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PREFACE 



Through this monograph the authors seek to inform the educational community, 
particularly vocational educators at the state ar local levels, of a potential resource for 
responding to additional demands for accou::^tability. That resource is the nation's 
unemployment insurance wage reporting system. The research presented herein focuses 
upon the use of this system by thirteen states to follow-up on former students in order to 
gain a picture of the post-education employment and earnings history. Administrators 
and others engaged in evaluation and accountability efforts will find this information to 
be useful in planning new initiatives to strengthen assessment, program evaluation, and 
federal reporting. 

In this report, "accountability" is used as a broad construct to encompass 
evaluation of policy, programs, and services at both the state and local levels. Recently, 
the new mandates of the 1990 Carl D. Perkins Vocational and Applied Technology 
Education Act Amendments, along with other educational reform initiatives, call for 
increased attention to the documentation of program and student outcomes to ensure that 
educational efforts are indeed achieving their intended impact Traditionally, 
accountability has focused on ensuring that state boards and local recipients of funds 
expend money in ways consistent with the purposes of the Perkins Act However, it is 
imperative that the vocational education community adopt a view of accountability which 
is broader in focus and examines student, program, and fiscal outcomes at levels ranging 
from the classroom to the state legislature. With a broadened perspective of 
accoxmtability, the prospects of developing a world-class American workforce, through 
better instruction as well as adequately funded and coordinated programs, ar^ 
significantly improved. 

This report illustrates several of the major efforts commissioned in recent years by 
state boards of vocational education to explore the feasibility and utility of using the 
unemployment insurance wage record system. These case studies in thirteen states 
provide a rich anthology of efforts to explore post-program employment, earnings, and 
other experiences of vocational education students in relation to programs in which they 
were enrolled. This study was not intended to be a comprehensive analysis of all efforts 
nationally regarding the use of wage records in vocational education. The design of the 
study sought to inform state and local officials regarding some of the most extensive 



effort in illustrating states to utilize Unemployment Insurance (UI) wage records. 
Understanding the efforts of various states is particularly valuable as tlie use of 
performance measures and standards in vocational education programs is inaugurated 
nationally. 

The first section provides a backdrop for those desiring more information 
regarding accountability, new challenges in performance assessment, establishment of 
performance standards systems, and what role the UI wage records might play in assisting 
vocational educators develop those standards and measures. Readers primarily interested 
in what states have done in their efforts relative to UI wage records are dixectt^d to the 
second section where state by state descriptions are presented and detailed. The third 
section presents an overview of what all the states accomplished and presents the 
findings, conclusions, and recommendations for future use. The redundancies are 
intended to facilitate reading of the report on the part of different individuals with varied 
interests. 

During the conduct of this research, concurrent efforts were underway by various 
individuals and organizations also interested in the UI wage record system. The authors 
wish to thank them for sharing information and offering advice. These include staff of 
the National Commission for Employment Policy, the National Governors' Association, 
the National Center for Research in Vocational Education (NCRVE), the National 
Occupational Information Coordinating Committee, and the U.S. Department of 
Education. Interested readers should watch for reports and handbooks to be published in 
tlie near future. 

Crucial to the success of the project were those persons in the states studied who 
provided so much of their time and talent to this effort Without their suppoI^ interest, 
and advice, neither the research nor this report would have seen the light of day. They are 
identified in Appendix D and our sincere appreciation goes out to them. 

Lastly, Robert H. Meyer and Robert P. Sorensen, bo?h of the University of 
Wisconsin at Madison, provided their expertise along with continual support and 
encouragement at all stages of the research. They also contributed greatly to the final 
review process. Additional reviews were obtained from Franz Backus of the Wisconsin 
Department of Industry, Labor, and Human Relations; Jay Pfeiffer of the Florida 
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Educational and Training Placement Information Program; David Stevens of the 
University of Baltimore; and a NCRVE unidentified reviewer. Their familiarity and 
knowledge of the UI wage records and their possible applications to performance 
outcomes contributed immeasurably to the conduct of the study and to the final report. 
We wish them well in their future endeavors. Although many people contributed 
significantly to our efforts, the authors take full responsibility for any errors and most 
omissions. 



s 

iii 



INTRODUCTION, BACKGROUND, AND PROCEDURES 



Introduction 

An integral aspect of the educational refom movement has been an intense interest 
in assessing the quality, productivity, and effectiveness of both current and new 
educational endeavors. In July 1990, following the declaration of the national education 
goals in September 1989, the National Goals for Education Panel was appointed and 
charged with assessing the nation's progress toward achieving the national goals. Moving 
toward these new educational outcomes requires policymakers and educators to have a clear 
picture of both the current levels of student achievement (i.e., a benchmark), as well as 
definitive goals for educational institutions and student achievement The establishment of 
new goals for the nation's learners requires new assessment strategies. 

Under the Constitution, education is primarily a state responsibility. Given the 
widely differing views on the aims and purposes of education and state and local resources 
committed to those aims, it is understandable why the nation lacks common national 
measures or standards for many educational phenomena. As the inaugural report of the 
National Goals for Education Panel (1991) conveys, there are many facets of specific goals 
about which we are still largely ignorant As the report suggests, a critical need exists for 
"state and national data on how well the knowledge, skills, and attitudes of American 
workers compare to those of workers in other nations, especially those that are our prime 
economic competitors" (p. 15). 

Among the various strategies that should be considered for improving educational 
accountability and its impact on the workforce is the use of a wage record database 
(Crosslin & Stevens, 1989). Since 1988, state governments have been required to collect 
on a quarterly basis the following information from private sector employers: gross 
earnings (by social security number), industry of employment (by Standard Industrial 
Classification), and county of employment (However, many states had wage reporting 
systems prior to 1988. In fact, in 1984, only eight states did not do wage reports.) The 
use of social security numbers in this data system is potentiaUy an invaluable resource for 
tracking students' experiences and success in the labor market When compared with other 
longitudinal databases such as the Census, federal or state Internal Revenue information 
(Strong & Jarosik, 1989), state-level higher education databases (Smith, 1989), or national 
longitudinal studies in education (Hoachlander, 1989b), a wide array of important policy 
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questions can be asked about the performance of former vocational education students, 
both graduates and leavers. The potential high quality of the data included in the UI wage 
records and the significant cost savings in obtaining the information (compared to high 
costs of traditionaT graduate and employer follow-up surveys) make careful consideration 
of this option particularly important 

In response to the growing national interest in the UI wage records to address 
vocational education accoimtability concerns, this project focused on the following three 
objectives: 

1 . To describe the efforts of selected state boards of vocational-technical education that 
have developed and experimented with strategies for accessing, analyzing, and 
reporting the UI wage record data. 

2. To describe the substantive and technical issues associated with tlie database (e.g., 
coverage and content of the wage record data, accuracy of the records, privacy 
concems, useful analyses of employment and educational data). 

3. To prepare a technical report providing a summary of the project procedures and 
findings, including recommendations for national, state, and local administrators 
and policymakers. 

Background 

Accountability 

Accountability is pervasive in most discussions of American education today and 
vocational-technical education is not exempt from the dialogue. Legislators, policymakers, 
tax payers, and educators themselves are calling for increased accountability for all social 
programs. Vocational educators have long attempted to "prove the worth" of their 
programs in a variety of ways. Today, the demand for that accountability will require more 
coordination and greater leadership from federal and state officials, as well as from the 
vocational education community. McDonnell and Grubb (1991) have noted the following: 

The growing emphasis on accountability in all aspects of American life, 
from the on-time departures of airlines to hospitalUy rates, is likely to make 
performance standards an increasingly prominekit feature of education and 
training policies As a result, the challenge for future policy design will 

11 
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not be to craft a single instrument intended to eliminate slippage on a few 
narrowly defined measures such as program cost and short-term job 
placement, but to combine a variety of instruments in ways that will create 
incentives for local education and training institutions to advance the goals 
of preparing individuals for productive employment — ^including widespread 
accessibility of services and the acquisition of skills that enhance long-term 
job opportunities, (pp. 53-54) 

These goals for vocational education are important in the consideration of 
accountability and in the formulation of public policy. Traditionally, vocational education 
programs have utilized training-related employment (i.e. Job placement) as evidence of the 
effectiveness of a given program or institution. However, it is also recognized that such 
concepts are not always u: eful. Several studies have noted that demographic factors 
associated with various screening or sorting methods of students for entry into the wide 
variety of work-related education and training programs account for many of the 
differences in postprogram employment status, job retention, and earnings enhancement 
(McHugh & Stevens, 1989, p. 34). 

In secondary schools, the current educational reform movement suggests that all 
students need to be adequately prepared to compete in a global, technologically intensive 
economy. The recent report of the Secretary's Commission on Achieving Necessary Skills 
(SCANS) (1991) contends that all high school graduates need expertise in using resources 
. -d information effectively, manipulating systems and technology, and using expanded 
interpersonal skills such as problem solving and teamwork. Developing new and more 
precise measures of these skills in work settings will be crucial in evaluating the effects of 
both vocational education programs along with more general education enterprises. 
Moreover, the philosophies of programs and of administrators shift and vary so that 
employment may or may not be a significant indicator of success in all cases: 

Vocafional educators, and to a lesser degree providers of employment 
training, have often objected to the exclusive focus on labor market 
outcomes. A variety of factors beyond their control affect labor market 
outcomes. Hence, they argue, holding them accounu»ble for employability 
rather than employment would be more appropriate for judging program 
success. Employability means having the necessary skills to perform 
effectively on the job, regardless of whether employment opportunities are 
immediately available. Consequently, assessing employability requires 
attention to a set of learning outcomes, for which service providers can 
assume greater responsibility. (Hoachlander, 1989a, p. 3) 
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Lastly, others would argue that for vocational education to be truly accountable it 
must be viewed within the larger context of education and society and not as an isolated 
program reporting a placement rate of "72% six months after graduation," The 
contributions of vocational education to overall educational attainment is an important 
consideration. As David Stem (1988) notes: / 

In using data on employment or earnings to measure aggregate outcomes of 
Postsecondary Vocational Education (PVE), there should be some attempt 
to estimate net effects. This means trying to estimate how many non- 
participants in PVE would have been placed in jobs taken by PVE 
graduates, and what happens to the non-participants' employment, 
earnings, and expressed job satisfaction. In attempting to measure how 
much PVE improves access of under-represented groups to certain 
occupations, it would be desirable to find out whether people displaced by 
PVE graduates are themselves members of under-represented groups. 
Failure to make such corrections seriously reduces the value of labor market 
outcomes as indicators of program success, (p. 14) 

All levels of public policy development directly affect vocational-technical 
education. Hence, it is important for vocational-technical education to recognize the 
demands for accountability at the federal, state, and local levels and meet them head-on: 

lawmakers will continue to respond to the nation's changing needs through 
targeted spending. Congress can achieve greater economy and equity in 
programs and improve its own stewardship of public resources if it chooses 
targeting language with appreciation of the capabilities and possible 
limitations of existing data at the national and local levels and with 
recognition of the periodic need to add new elements to data sources. It can 
make g'-eat strides in rationalizing the policymaking and policy 
implementation process if members begin now to construct a better labor 
market information system for the future. (Stevens & Duggan, 1988, p. 

Assessing Educational Performance 

Foremost among the changes called for both within and outside of educational 
circles in tiie name of accountability is the establishment of performance assessment 
systems for education. Rather tiian evaluations based upon educational inputs and 
processes, system performance or outcomes are seen increasingly as the key to educational 
reform and improvement. The Total QuaUty Management in tiie private sector places 
particular emphasis on program outcomes or the "bottom line." Many educators translate 
this approach as requiring extensive student assessment as a basis for improving the 
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accountability, management, cost effectiveness, and ultimate performance of public 
programs* 



Many vocational educators would argue that performance monitoring of outcomes 
has occurred throughout the history of vocational education and point to the use of job 
placement rates as the prime indicator. However, the concept of performance standards 
and assessment goes well beyond this premise: 

While there is much variation in the way the States have approached the 
design and management of change, overall "accountability" has become a 
guiding principle. Schools are expected to act like businesses, and account 
for their successes and failures; and though it has never been very easy to 
apply quantitative measures to complex education^il processes and 
outcomes, more and more school systems have been doing just that In 
many places, performance measurement has become a basis for incentives 
or sanctions directed at various levels of the educational '—stem. For 
example, merit pay for teachers is an attempt to reward teachers for superior 
performance, while so-called "bankruptcy" laws penalize school districts for 
poor performance by transferring their assets and liabilities. (Office of 
Technology Assessment, 1989, p. 11) 



While this study addresses the issue of performance from an administrative 
perspective, there are multiple uses for the performance information currently being 
comidered. According to Brown and Choy (1988), the quality of information available to 
consumers (i.e., parents and students) who wish or need to make educational program 
choices has become a critical issue as well: 

Disclosure of information on outcomes could protect students from the kind 
of abuses discovered in the GI educational allowance and Guaranteed 
Student Loan programs. If students were provided with outcome 
information such as completion rates, job placement rates, and entering 
wage rates for different programs and institutions, they could avoid 
programs that offer them little (return) for their investment of time and 
money, (p. 4) 



More information on student outcomes from selected programs could help students 
make the best choices for themselves from available options. Currently, students are 
required to choose schools with limited objective information about the quality of programs 
and support services such as job placement. Program and institutional evaluation results 
and other information collected by the state or individual institutions are rarely available to 
the public. 



To date, performance-based approaches to accountability have been utilized in tlie 
Job Training Partnership Act (JTPA). Since the mid-1980s. Congress has seen the JTPA 
as an experimental opportunity and is now extending the use of performance measures and 
standards to a number of federal social and education programs, including welfare, adult 
education, and vocational-technical education. Several insights regarding the 
implementation and use of performance assessment systems have emerged from the JTPA 
experiences. Most notable is the concern that the use of performance standards tends to 
create "creaming" in the selection of participants. In order to assure reaching outcome 
standards, there is a natural inclination to enroll only those students who are most likely to 
show high levels of attainment at the end of the program. Apling (1989) offers the 
following additional insights from the JTPA experience: 

• the influence of performance standards on the type of training 
provided with a danger that effective — but long term and 
expensive — ^services will be discouraged in favor of short-term and 
inexpensive approaches; 

• the difficulty in meeting multiple standards, some of which may not 
be mutually compatible; 

• the problem of adjusting standards for programs in different labor 
markets or serving different types of clients; and 

• the difficulty in setting minimum standards. 

Experience from performance standards in other program areas indicate that, 
unless carefully designed and implemented, standards may not improve 
performance and may have undesircd effects, (p. 18) 

The use of performance-oriented approaches to achieve accountability is 
multifaceted. The challenge confronting educators, and vocational educators in particular, 
is one of devising a set of performance measures and standards which provide useful ways 
of monitoring the federal, state, and local investments in rapidly changing vocational- 
technical education programs. 

Vocational Education Performance Standards 

The recently passed Carl D. Perkins Vocational and Applied Technology Education 
Act Amendments of 1990 requires each state receiving funds under the Perkins Act to 
develop and implement a statewide system of core standards and measures of performance 
for secondary and postsecondary vocational education programs (Sec. 115[a]). Among the 
core standards and measures addressed in the Perkins Act is "placement into additional 
training or education, military service, or employment" (Sec. 115[b][2][D]). As mentioned 



earlier, vocational education has long concerned itself with "job placement" as an outcome. 
However, the issue of placement needs to be revisited in the light of accountability, 
performance standards, and educational reform: 

Vocational educators object to evaluating their programs on short-term 
placement and employers* initial satisfaction, because PVE (postsecondary 
vocational education) is righUy intended to give students a variety of skills 
that will increase their productivity as their careers advance. Chasing job 
vacaiicies may distract PVE programs from their public purpose of 
improving employees* productivity. The tradeoff between achieving high 
placement rates and improving long-term productivity is more acute if 
placement is supposed to be "related to training" because, if PVE programs 
focus on giving their graduates an edge in competing for a narrow range of 
jobs, tiiey will presumably emphasize a narrow set of skills at the expense 
of more generally applicable ddlls and knowledge. More narrow training 
may not make PVE graduates as productive in the long run if they do stay in 
jobs related to their training and, by definition, it will not pay off as much as 
broader training if they take jobs in other fields. (Stem, 1988, p. 13) 



Just as the above refers to postsecondary vocational education, similar arguments 
are advaiiced for vocational education programs at the secondary level such as the 
following observations by Hoachlander (1989a): 

the single-minded focus on employment outcomes is inappropriate. While 
labor market outcomes are a necessary indicator of the effectiveness of 
vocational education, they are not sufficient Many factors other tiian the 
quality of a particular program affect a student's employment history. 
Economic conditions, family formation, personal interests, motivation, and 
alternative opportunities all figure into what students do after participatmg in 
vocational education. Under the right economic conditions, for example, 
graduates of very poor programs may all do very well in the labor market, at 
least for a few years, (p. 16) 



Given the constraints of job placement outcome measures, die search for accurate, 
reliable measures and standards for evaluating vocational education continues. However, 
tile development of labor market outcomes remains a critical issue. According to 
Hoachlander (1989b), 

The major difficulty in using labor market outcomes to assess program 
effectiveness lies in isolating outcomes tiiat may truly be attributed to tiie 
effectiveness of a program from tiiose tiiat result from factors beyond tiie 
control of the program. Two obvious external factors are general economic 
conditions and student characteristics. If completers of different programs 
face different economic conditions, comparisons of unadjusted labor market 
outcomes will not yield accurate comparisons of the relative effectiveness of 
the programs. Similarly, programs tiiat serve students witii different 
characteristics may realize different labor market outcomes that have littie to 
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do with program effectiveness. For example, a (skilled) trade program that 
serves mainly women may have lower placement rates than a similar 
piogram that serves mostly men. This is not because the program is 
inferior, but because its graduates continue to face discrimination in the 
labor market Evaluations of program effectiveness in terms of labor market 
outcomes, therefore, must distinguish programmatic effects from outcomes 
that result from factors beyond the control of the vocational education or 
employment and training program. The inclusion of good statistical 
controls will help to achieve this distinction, (p. 5) 



Data collection on outcomes is of primary concern to those responsible for the 
development of systems of performance and evaluation. However, acquiring high quality 
outcome data from existing information sources is problematic. 

For rigorous, mid- to long-term follow-up on the impact of participation in 
vocational education, the national longitudinal studies will remain the best 
sources of data. They cannot be topped for detail over time on student 
characteristics, education, family formation, and employment history. The 
primary problems with the longitudinal studies stem from sample size. It is 
not possible, for example, to conduct follow-up for small to medium sized 
programs. Nor is it possible to make comparisons among states or types of 
delivery systems. (Hoachlander, 1989a, p. 28) 



Data collection within states causes some additional, special problems for vocational 
educators interested in placement outcomes: 

While there is growing evidence that appropriate performance criteria can be 
defined, measured, and linked to funding, obtaining accurate, timely, 
inexpensive data has posed major problems. Employment training 
programs, because they work with smaller numbers of clients than 
vocational education and because they have limited follow-up to relatively 
short periods following program completion, have successfully collected 
data on placement Large scale, longer term efforts at tracing the labor 
market outcomes of participants in either postsecondary vocational 
education or employment training have not been successful. (Hoachlander 
1989b, p. 32) 

Further, in most states the follow-up job placement is not considered the primary criterion 
for a secondary vocational education program. 

However, a potential side benefit of successful implementation of performance 
standards in vocational education might be closer relationships with employment training 
programs. As Butler (1988) notes, it may be possible if vocational education continues to 
move in the direction of outcome-based management: 
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for the two systems (vocational education and JTPA) to become more 
integrated with each other. Those of us who work in one part of the field or 
the other may understand the distinctions, but the general public does not, 
and certainly Congressional and state legislators who ultimately make the 
policies and vote on whether and to what extent to fund them are not 
comfortable that one hand knows what the other is doing. Moving both 
systems in the direction of outcome-based programming and management 
may bring the two closer together to the benefit of both. (p. 22) 



While developmental problems certainly appear in the forefi^ont when discussing the 
impact of performance standards in vocational-technical education, one must not lose sight 
of the local nature of the implementation of such a system. Full utilization of the results of 
the system and systemic program improvement will only result when all facets of a 
performance-oriented system are incorporated fully by administrators and professionals 
(i.e., "buy-in occurs"). 



Historically, the vocational education community has been called to respond to 
various accountability demands. Because such a significant share of the resources and 
conceptual direction for vocational education programs is found in federal legislation, the 
accountability expectations for these "special" funds have always been high in the minds of 
employers, taxpayers, and policymakers. Performance standards are another and maybe 
the last in this long history of federal mandates: 

while the collection of vocational education data has been beset by a number 
of false starts with unsatisfactory results, it is important to keep in mind that 
Congress has asked much more about vocational education than any other 
component of equal size in the secondary and postsecondary systems. 
Questions that on the surface would seem straightforward have, in fact, 
been quite complex. Had they been asked of other parts of the secondary 
and postsecondary curriculum, they would have proven equally difficult to 
answer. It is important to remember, therefore, that the problems that have 
plagued the collection of data on vocational education are not unique to it; 
rather, they are endemic in the larger systems we use to organize and 
account for secondary and postsecondary education. (Hoachlander, 1989a, 
p. 35) 



However, in response to the call for increased educational accountability and the 
establishment of perfonnance standards for vocational education, the identification of cost- 
effective labor market outcomes remains a key issue. A recent study by the Office of 
Technology Assessment describes several of the possibiUties that now exist for collecting 
and analyzing new information which will make the establishment of peri^ormance 
standards both feasible and cost effective. 



Recent improvements in the quality and cost of data collection and storage, made 
possible by mandated changes in employer reporting of wage and earnings data, provide 
partial relief for these concerns (e.g., validity of follow-up information provided by 
students, high cost of conducting follow-up studies) and increase the feasibility of 
designing cost-effective labor market outcomes indicators. In particular, the use of wage 
records maintained by the states in compliance with recent amendments to the Social 
Security Act could be a first step toward improved labor market indicators of program 
performance. The wage records data is more accurate than self-reported survey 
information, allows for longer-term evaluations of employment, and can be merged with 
other data such as military records and computerized school transcripts. 

There remain some important technical issues to be resolved before the wage 
records system could be implemented for secondary vocational program evaluation. First, 
school transcript information is nut uniform, despite recent efforts to standardize vocational 
course definitions and curricular offerings. In addition, clarification of confidentiality 
restrictions pertaining to individual financial data, and the decision to maintain long-term 
archives of earnings and employment data, would be minimal prerequisites to 
implementation of wage records system for performance evaluation (Office of Technology 
Assessment, 1989, pp. 4-5). 

Unemployment Insurance Wage Record Data 

Each state requires civiliiai employers to report on a quarterly basis, and under 
various names (e.g., wage records, unemployment compensation records), earnings of all 
employees (together with their social security number). (A detailed description of the 
database is included in Appendix A of this report.) This database, for all practical 
purposes, serves as a quarterly census of wage records. In addition to its principal use, 
that is, in the administration of the unemployment insurance or compensation laws, other 
uses have evolved. Some states operationalize this employer "universe" for wage surveys 
undertaken for labor market information. Use of this database to track completers and/or 
leavers of specific social programs, including job training and education, has been 
discussed for several years. Indeed, testing of the system for these secondary purposes 
has taken place and considerably more attention is focused on potential uses of the 
database, specifically to assess student outcomes of vocational education. 
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One of the most prominent examples of a federal program which could employ 
wage record data to improve accountability is vocational education. The Perkins Act states 
that the objective of vocational education is to enhance the productivity £ .id labor market 
success of its students. Serious consideration mxist be given to establishing and funding an 
adequate and functional accountability system for vocational education programs. One 
major purpose of such an accountability system would be to induce a reconsideration by the 
states of the way they fund local vocational education programs (Bishop, 1989, p. 19). 



For many^ the traditional measures of performance outcomes (i.e., follow-up 
studies of graduates) simply do not pn>ve satisfactory: 

The major barrier to implementing performance measurement in vocational 
education and in other programs has been the unreliability and 
inappropriateness of the performance measures currently available. 
Training-related placement rates, currently reported to state department of 
vocational education (in some states), are not comparable across districts 
and programs. In addition, they are subject to manipulation aiid suffer from 
a nonresponse problem. Archived UI wage record data wovJd be extremely 
valuable here. Estimates of the earnings impact of vocational versus general 
education can be obtained rather easily by merging wage record data with 
school files on curriculum, grades, and the test scores of students. 
(Bishop, 1989, p. 20) 



To date, most of the exploratory research associated with the Unemployment 
Insurance (UI) wage record database has focused on the labor market performance of youth 
adults completing programs funded by JTPA. A recently completed feasibility study of 
using UI wage records to evaluate JTPA revealed tlie following: 

• The use of UI wage-record data would enable detailed analyses of the 
long-term employment and earnings patterns of JTPA terminees. 

• UI wage-record data offer great promise as a resource for studies 
designed to measure the net impact of employment and tr^^Jiing 
programs. 

• UI wage-record data is a cost effective means for obtaming pre- and 
postprogram employment and earnings infonnation on JTPA 
participants. (Baj, Trott, & Stevens, 1991, p. ii) 

This same study suggested that four major issues must be addressed in the strategic 
decision at the national level whether to use the UI wage record system as a tool for 
monitoring and evaluating training programs. These issues included technical feasibility 
and costs, both of which have proven to be adequately addressed in pilot studies to date. 
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Procedural differences among states could be reconciled through Congressional leadership. 
Finally^ the "political will** to stimulate change and innovation is also a federal 
responsibility (Baj et al.» 1991, pp. 95-96). 



Use of the UI wage record data provides an unique opportunity to do studies of a 
longitudinal nature, which heretofore have been extremely difficult and costly to imdertake 
in education. Over time, this data could provide some extremely useful information on the 
effects of vocational-technical programs on numerous labor market behaviors. According 
to Brown and Choy (1988), among the important effects and variables to examine are (1) 
longitudinal earnings and employment profiles, (2) job mobility within industries and 
states, and (3) pre- and postprogram earnings (p. 23). 



There is also a significant potential for researchers or policymakers to create 
comparisons groups through statistical manipulations with Census data and hence have an 
additional vehicle by which to assess the performance of vocational education. Bishop 
(1989) provides an example of a potential analytical model for assessing the impact of 
vocational programs: 

A year after the student leaves school, and periodically after that, wage record data 
on employment, quarterly earnings, establishment size, and industry could be 
merged with the confidential record containing measures of the student's academic 
performance prior to entry into vocational education. An analysis would be 
conducted of this data and a model developed predicting the earnings of those not 
attending college. The computer could then generate a predicted earnings and 
employment profile for each noncollege-bound student based on the actual 
outcomes of the high school's noncollege-bound students who did not pursue a 
vocational curriculum. The mean difference between these predictions, and the 
employment outcomes realized by the school's vocational students would one of the 
factors considered when assessing the effectiveness of a school's vocational 
program. Mean differences would be calculated for students grouped by the 
number of courses taken, by field of specialization, by whether the student 
graduated and by grades in vocational courses, (p. 20) 



Although the development and utilization of the UI wage record data is a function of 
state govemment, some researchers have called for a national archiving of the data so that 
national studies could more easily be conducted. This concern would certainly be 
applicable to the concerns of vocational-technical education as well. However, since the 
Perkins Act requires states to develop their own systems of performance standards, states 
may discover unique advantages in using the UI wage record system of their own state, 
foregoing interstate comparisons: 
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While there is clear interest nationally in determining effective policy and 
program approaches, the need to act on the information is more pressing at 
the state level. This particularly true in education. Archived, quarterly 
employment and earnings data would permit states to evaluate program 
effectiveness across programs, including vocational education, job training, 
job placement, and others. . • . Further, as states become more involved 
with customized and industry-specific training — under JTPA Titles n and 
in, as well as vocational education — ^they would be able to document the 
impacts of each over time, compare public and private sector effectiveness 
with various approaches and make appropriate program modifications. 
(King, 1989, p. 7) 

State agencies responsible for managing the UI wage record information are sharing 
this data for a variety of reasons, most unrelated to education and training. Forty-one states 
responded to a 1988 survey initiated, conducted, and compiled by the Wisconsin 
Department of Industry, Labor, and Human Relations, Unemployment Compensation 
Division (1988), which attempted to secure information regarding individual state's data 
sharing efforts of its UI wage record reporting system. Ninety-three percent or thirty-eight 
of the responding states indicated that they share wage record data with other state or 
federal agencies, those most often cited being the state agency responsible for Health and 
Social Services, the Internal Revenue Service, and the State Department of Rex/enue. Four 
states reported sharing this information with education agencies (p. 1). 

Rising costs and concerns regarding confidentiality and security are critical 
concerns for state agencies as data sharing requests grow. Another potential inhibitor to 
expanded utilization of the wage record data for use in perfonnance outcome measurement 
is the lack of imderstanding regarding the appropriate use of the data. 

Discussions among state agency personnel at the Northeast-Midwest Institute LMI 
Forums revealed a tension between data producers who are not funded to offer extensive 
technical assistance to data users. The latter group includes LMI consumers who are not 
able to spend the time that would be necessary to acquire a working understanding of the 
technical aspects of the data sources. This creates an understandable caution on the part of 
LMI producers, who are reluctant to release information that they suspect will be misused. 
Simultaneously, frustrated consumers complain about the lack of cooperation exhibited by 
the producers. In a paper presented to a recent national LMI annual meeting, David 
Stevens (1989) argues, 

There is a critical need for capacity building in both the producer and 
consumer arenas. It is penny wise and pound foolish to consciously allow 
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the quality and availability of information to deteriorate at the very time 
when target efficient management in botii the public- and private-sectors is 
required, (pp. 7-8) 



Duggan (1988) offers the following advice for state and local education 
policymakers on this issue: 

• Understand what LMI reveals about the directions of the state's 
economy and the training and education needs of the current and 
future work force. Learn how to use LMI in education 
policymaking. 

• Give serious thought to the weaknesses in data conceptualization on 
the "supply" side of the employment equation. Design education 
data programs that will earn die same reputation for statistical quality 
and reliability as the BLS programs, (p. 29) 

Utilization of the UI wage record information data holds promise for improving 
accoimtability in vocational education. Some states are already tapping this resource — as 
wiU be revealed in the balance of this report While perhaps not a panacea, the UI wage 
record files represent a tool for vocational education leaders and policymaking in evaluating 
and improving programs. As Hoachlander (1989b) notes: 

In short, the unemployment insurance files provide a promising low-cost approach 
to obtaining current, accurate, and fairly comprehensive data on some selected labor 
market outcomes for students participating in vocational education programs. The 
approach is far from perfect and would benefit fix)m a few modifications that do not 
appear to be burdensome or expensive. Certainly, the information and coverage is 
far superior to the follow-up efforts attempted by the Vocational Education Data 
System. While this method lacks the precision and detail of the larger national 
longitudinal studies, it provides an economical alternative for obtaining institutional 
and statewide data on an annual basis. Thus, potentially good data is available on 
labor market outcomes for participants in postsecondary vocational education and 
employment training programs, (p, 37) 
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Procedures 



As noted earlier, this study sought to describe the pilot and demonstration efforts of 
thirteen states that had utilized the UI wage records in evaluating some aspect of their 
vocational education program The strategies employed in this study included conducting a 
comprehensive, ongoing literature review; a brief questionnaire survey mailed to state 
directors of vocational-technical education; summary and analysis of information provided 
by thirteen states; and in-depth telephone interviews with key state agency personnel 
responsible for conducting the activities and studies linking vocational education data with 
the UI wage record data system. 

Over the past two years the literature on the use of wage record data has become 
voluminous. Much of this literature has been generated by various groups associated with 
implementation of JTPA programs. Several initiatives have been undertaken by states 
using the UI wage record data to assess the impact of JTPA programs. The National 
Governors' Association (1992), among other groups, have urged the U.S. Department of 
Labor to measure the labor market performance of former JTPA trainees. In addition to 
other sources, the National Assessment on Vocational Education (NAVE), which was 
completed in 1989, commissioned a series of papers which examined several alternatives 
for improving the assessment of labor market and further education and training oi comes 
of former students, including tracking students through computerized data systems. 

In January of 1991, a brief memorandum was mailed to each state director of 
vocational-technical education. The directors were asked to indicate whether or not efforts 
in their state had been undertaken to explore, develop, or conduct research on student 
follow-up utilizing one or more of the following automated databases: (1) UI wage 
reporting, (2) income tax data, (3) social security, or (4) social services. State officials 
were also asked to return any reports or written materials that would describe their efforts 
in this arena. 

Of the fifty-three states and territories, twenty-six (49%) responded affirmatively, 
indicating they had or were examining the UI wage record database as a tool for evaluating 
the labor market effects of vocational-technical programs. ^ Overall, a total of thirteen states 

Hn a survey recenUy conducted by the NaUonal Center for Research in Vocational EducaUon at the 
University of California at Berkeley, nine stales on the secondary level and thirteen slates on the 
postsecondary level reported having used unemployment insurance data in the past to report on student 



15 O " 



were identified as having conducted recent studies which involved the UI wage record 
database. Since responses were sought only frvom those states that had undertaken 
significant studies (i.e., an effort which, at minimum, had produced a report), it was 
assumed that nonresponding states had not leaciied an in-depth level of consideration. 

The reports, memoranda, and related information submitted by each of the states 
were reviewed and analyzed by members of the research team. Most of the projects 
reported by the states were feasibility studies intended to examine the utility of using the UI 
wage record data system for tracking employment, earnings and whether the employment 
taken by former students was training-related. From the information provided, a two- to 
three-page state profile was prepared describing each state's initiative. The state case 
profiles, which are presented in the following chapter, describe the purpose and parameters 
of the investigation, the employment and educational data used, the analyses performed, 
some of the primary findings, and the major conclusions and recommendations emerging 
from the project 

As the state case profiles were being compiled, a detailed interview protocol was 
developed. The purpose of the in-depth interview was threefold: (1) to obtain a review of 
the case profile by a knowledgeable state official to ensure that it was accurate and 
complete, (2) to obtain perspectives on any future plans for using UI wage record data in 
vocational education accountability systems (e.g., the system of performance measures and 
standards required under the Perkins Act Amendments of 1990), and (3) to gain detailed 
information on a series of critical issues surrounding the use of the UI wage record data 
(e.g., assurances of confidentiality, reliability and validity of the UI data, crosswalking 
employment codes with curriculum/program data, and the availability of uniform data on 
vocational education such as completers and programs). The telephone interview protocol 
is presented in Appendix B. 



perfonnance. Additionally, sixteen (secondary) and nineteen (postsecondary) stales respectively reported that 
they were considering using the data source in the future. See Hoachlander and Rahn (1992, A-1, A-2). 
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STATE PROFILES 



The thirteen states addressed in this study are presented in the following 
classifications: 

Group 1. Ongoing Systemic Utilization of UI Wage Record Data 

Group 2. Feasibility or Exploratory Work Completed/Planning Undervv':ay for 
Systemic Use 

Group 3. Feasibility c r Exploratory Work Completed/Future Development or Use 
Uncertain 



Ongoing Systemic Utilization of UI Wage Record Data 

Four states have been identified as having made major strides in the use of UI wage 
record relative to vocational education programming. Florida is a pioneer in this area and 
maintains the most sophisticated system. Washington's Community College System has 
developed innovative uses of the UI wage record file information and has cooperated with 
neighboring states in securing additional wage records. Alaska actively promotes the 
utilization of its UI wage records among a variety of educational and training programs and 
provides services upon request Finally, Colorado has been building up a longitudinal 
profile of its graduates and shares the information with its colleges. 

Florida 

Florida is indeed a pioneer in computer matching of data files. In addition to the UI 
wage record files, it matches student records with higher education, state administration 
corrections, and the following federal agencies: defense, personnel management, and the 
postal service. Florida state law requires compliance with a certain placement rate and also 
establishes the Florida Education and Training Placement Information Program. It has 
been demonstrated that the Florida system is relatively inexpensive. 

Clearly, a pioneer in the utilization of automated databases to assist in the 
assessment of educational programming, the Florida Education and Training Placement 
Information Program began as the Occupational Identifier Project. It was formed in 1984 



both under a legislative directive and a joint agreement between the Horida Department of 
Education and the Florida Department of Labor and Employment Security and supervised 
the by the State Job Training Coordinating Council with the Florida Advisory Council on 
Vocational Education serving in an advisory capacity. The project was developed to 
address a legislative directive enacting a placement standard for public vocational education 
job preparatory programs. The standard requires that sevei ty percent of all program 
completers obtain employment which is related to their training. "Training-related" is 
defined as conditions in which former students 

• obtain employment utilizing skills or competencies acquired through training; 

• continue their education; or 

• enlist in one of the uniformed nulitary services of the United States. 

The 1989 Florida Legislature established the Florida Education and Training 
Placement Information Program as the Department of Education's primary source for the 
collection of student follow-up information, especially with respect to collecting the data by 
computer matching. The program provides comprehensive, reliable, and cost-effective 
follow-up data on former Rorida students through civilian employment, continuing 
education, and military data via automated databases. 

The Placement Information Program relies on the cooperation of Florida's State 
Departments of Education, Labor and Employment Security, Administration, and 
Corrections, as well as tiie following federal agencies: Department of Defense (former 
students enlisted in tiie military). Office of Personnel Management (former students 
employed by the federal govenmienfs career service system), and die U.S. Postal Service 
(former students employed with the postal service). 

Three policy issues are addressed by Florida's Placement Information Program. 
First of all, as cited earlier, Florida statutes require tiiat seventy percent of vocational 
education program completers are placed in training-related situations. The program 
provides die accountability information to both state and local decision makers to address 
the standard. Second, prior to the establishment of die Placement Information Program, 
many diversified requests were made of botii employers and former students for follow-up 
data. This program consolidates such data collection activities utilizmg consistent collection 
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standards and definitions. Third, it is a goal of the Placement Information Program to 
develop a system of consistent core measures and standards for any public/private program 
where education and employment goals are outcomes. 

The advantages that the Florida Education and Training Placement Information 
Program identifies for this crossmatching system are 

• comprehensive — Consolidated data collection eliminates the need for former student 
employment surveys and studies done at public expense; minimizes paperwork, 
and, consequently, reduces employer contact and cumbersome surveys; 

• relatively inexpensive — With the program's approach using record matching to 
obtain information, the cost for each item of information decreases per unit as one 
sedcs more information; 

• very accurate — ^Placement Information Program data actually reflect what a former 
student was doing at the point in time used as a reference; and 

• comparable with other data resources — The program's approach uses the same 
program and occupational nomenclature used in other information resources. 

In its latest annual report (August 1990), a total of 196,665 former students 
enrolled during 19884989 were studied. These former students completed job preparatory 
vocational education programs in district secondary and postsecondary schools, community 
colleges, and state correctional institutions. The database mcludes former students in other 
selected educational and training programs such as migrant, exceptional education, 
displaced homemaker, single parent, and law enforcement education programs. Sixty-four 
percent of these former students were employed in over forty-two tiiousand Florida 
businesses, twenty percent were continuing their education as either part- or full-time 
students, and two percent were employed with the federal government or in the military. 

The same annual report also displays a work flow chart (see Graph 1) of the Florida 
Education and Training Placement Information Program which graphically describes all the 
components and activities associated with the program. 
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Table 1 
Fiorida Profile 



Authorization/Rationale 


Horida legislative directive enacting a placement 
requirement for public vocational education programs (job 
preparatory) 


Cooperating Agencies 


• Florida Department of Labor and Employment Security 

• Florida Department of Education 


Specific Data Files Used 


• UI wage record data 

• Databases of U-S, Department of Defense, U.S. Office 
of Personnel Management, U,S, Postal Service, State 
university system, coimnunity colleges, public schools 
(adult education), and state corrections 


Vocational Education Target 
Group/Size 


Annual summary of vocational education graduates/1988- 
1989 graduates n=196,665 which includes all secondary 
and postsecondary graduates in school and institutions 


Demogr^hic Data 


Race, gender, handicapping condition, age, and 
socioeconomic data 


Educational Data 


Institutional base of program (e.g., secondary school 
district, community college), vocational program, 
completer/leaver/special student status 


Employment Data 


Occupational code, number of weeks woriced, amount 
earned, coimty location, industry of employment, size of 
employer, employment in federal government (personnel, 
military, postal service), enrollment in university or 
community college 


Analyses Conducted 


Publishes earnings based on full-time, full-quarter 
employees. Also reports the percentage of people who 
were full-time, full-quarter employees from specific 
programs 


Major Findings 


64% of graduates were employed in Florida businesses, 
2% were federal employees or in the military, 20% were 
continuing their education 


Future Directions 


Maintain the program, investigate using neighboring states' 
wage record systems, and explore new ways of identifying 
"no find" students 


Format of Report 


• Mshed annual reports 

• Catalog of standard reports 

• Targeted studies 
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Washington 

Washington has expansive student records at the postsecondary level It matches 
these records with its UI wage records together with those from Alaska, California, Idaho, 
and Oregon. Matches are made before, during, and after enrollment identified those who 
are upgrading skills rather than initial preparation skills. State staff holds the position that 
confidence is higher for conclusions drawn for small subgroups: groups that typically do 
not respond to mail surveys. Washington has also done considerable work with an 
"employment related to education" indicator. 

In 1990, the Washington State Board for Community College Education 
(WSBCCE) (since renamed the Washington State Board for Community and Technical 
Colleges) tested the effectiveness of using other agencies' administrative data files to 
provide information on employment and educational outcomes of vocational education 
completers as an alternative to the traditional follow-up survey process. For reasons of 
economy and accuracy, the WSBCCE sought to substitute for its traditional mail or 
telephone follow-up survey of its former students a new technique to gather employment 
outcomes. The other agencies and the files included the Department of Employment 
Security (DES) (unemployment insurance and benefits systems) and the Office of Financial 
Management (OFM) (records of students enrolled at four-year public and private 
institutions). Since completion of the pilot study, WSBCCE has added the Department of 
Defense file as well as the UI files from the states of Alaska, California, Idaho, and 
Oregon- Agreements were entered into with both the DES and the OFM to facilitate the 
process of matching vocational education completers with those identified in the DES and 
OFM files. 

The social security numbers of 14,096 postsecondary vocational program and 
course completers of the school year 1987-1988 were matched against both the UI wage- 
record and the UI benefit history during the first quarter of 1989. Most of the completers 
were about nine months out of the programs. 

Information included within the student record for each completer arc the following: 

1. Community College Attended 

2. Ethnic Origin, Sex, and Age 
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3. Classification of Instructional Program Code (CIP) 

4. Handicap Status 

5. Limited-English Proficiency (LEP) Indicator 

6. Academic and/or Economic Disadvantaged Indicator 

7. Zip Code of Student's Current Address 

8. Institutional Intent Category Indicator (tells if the student took exclusively 
vocational courses, or vocational-with-academic, or basic skills/developmental 
courses) 

9. Enrollment History 

1 0. Exit Status (type of completer) 

Information provided by the DES to WSBCCE on each matched individual included 

1. Workforce Status (e.g., employed, unemployed) 

2. Standard Industrial Code (SIC) 

3. Employment Location 

4. Employment Related to Education Indicator (SIC of firm matches or does not match 
CIP code of completer's major) 

5. Estimated Annual Salaiy 

6. (Quarterly Hours 

7. Business Size (One of ten categories based on number of employees) 

8. Upgrading Retraining Indicator (Completer in one of tiiree categories: [1] worked 
for same firm before, during, and after training; [2] worked for same industry [SIC 
code] before, during, and after training; or [3] staited college before ES files were 
available) 
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A total of 9,387 matches were identified, or about sixty-seven percent of the original 
population of 14,096 postsecondary vocational education completers. 

Issues of concern identified by the Vocational Outcomes Research Review Group 
charged for the conduct of the pilot study included 

• Multiple employers of individual completers (16% of the completers had two or 
more employers during the match quarter so that the employment could be 
simultaneous or sequential) 

• SIC/CIP update regularity 

• Inclusion of all employees in ES files (exclusions, out-of-state workers) 

• Relationship of jobs to training 

• Definition of unemployment (insured versus uninsured) 

• Job upgrading and retraining 

When comparing the pilot test results to the more traditional mail and telephone 
surveys, the overall strengtiis and weaknesses of the DES matches were as follows: 

Strengths 

• The DES match process provides slightiy more information at considerably less 
cost and probably with less nonresponse bias. 

• Confidence may be higher in conclusions drawn for small subgroups since 
subgroups such as minorities tend to respond to a lesser extent to traditional 
surveys. 

• The jobs-related training information at the summary level is more reliable for future 
decision making since the ES match is probably a less biased source than a survey 
and provides information on more completers. 

• Because old survey processes usually limited a study to program completers 
because of the high costs, tiie pilot test allowed for more course completers to be 
included. 
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• The match of employment before, during, and after enrollment allows for 
identifying completers who are really upgrading job skills rather than preparing for 
anew job. 

Weaknesses 

• It is estimated that about twenty-five percent of employed completers were not 
matched. The WSBCCE plans to compensate for this by conducting a traditional 
mail and telephone survey of the nonmatching completers every five years. 

• The SIC to CIP relies on a protocol matching the SIC and CIP codes which has the 
potential for inaccuracies. This is because the protocol matches only the industry 
and not the occupation to the CIP code. For example, a graduate of a mid- 
management program employed as a secretary would probably show up as in a job 
related to training. 

• Since the "insured unemployment rate" is not the same as the more conmionly used 
unemployment rate, some misunderstanding can occur. 

Finally, the Vocational Outcome Research Review Group (see Appendix D) 
reached the following conclusion: 

The pilot study showed that SBCCE could use administrative databases to 
provide some of the information needed for outcomes assessment of the 
community college system. While the information is not as complete as 
might be desired, the coverage compares favorably with the much more 
costly survey process. The information available by data match also 
includes details which have not been provided in previous surveys such as a 
proxy measure for job upgrading and information on industry and firm size, 
(p. B-9) 



25 



Table 2 
Washington Profile 



Authorization/Rationale 


The Washington State Board for Community College 
Education sought to conduct a pilot study of the 
feasibility of using employment security and higher 
education files to track the employment and education 
outcomes of vocational education program 
completers. 


Cooperating Agencies 


Department of Employment Security, Office of 
Financial Management 


Specific Data Files Used 


• UI wage records (Washington, Alaska, California, 
Idaho, and Oregon) 

• Higher education student records 


Vocational Education Target 
Group/Size 


Postsecondary vocational education program 
completers for 1987-1988/n=14,096 


Demographic Data 


Age, gender, ethnicity, disability, LEP/disadvantaged 
indicator 


Educational Data 


Instructional program by CIP code, quarter/year 
started, quarter/year last enrolled, exit status, and 
other data 


Employment Data 


For the first quarter of 1989 — the workforce status 
(employed or unemployed), SIC code, employment 
location, estimated annual earnings, business size, 
hours worked during quarter, upgrade/retraining 
indicator, and employment related to education 
indicator (SIOCIP match) 


Analyses Conducted 


• Training related to industry code (SIC/CIP). 

• Upgrade^retraining indicator (whether employee 
worked prior to, during, and following training in 
the same firm and/or industry) 


Major Findings 


67% of completers were matched with the UI wage 
record database. 16% of the completers had two or 
more employers during the analysis quarter. The UI 
wage record match process provides slightly more 
information than traditional follow-up studies at 
considerably less cost and with significanfly less 
nonresponse bias. 


Format of Report 


Operations report available from the Washington 
State Board for Community and Technical Colleges, 
October 1990 


Future Directions 


Outcomes data is being reported annually in eighteen 
occupational program clusters at the state and 
institutional levels. Five-year averages are developed 
annually for employment, earnings, and other 
outcome variables. 
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Alaska 

Alaska state law requires employers to report job titles on the UI wage record The 
Alaska Department of Labor actively markets its matching capability and uses UI matching 
for a variety of training programs including vocational education, Job Opportunities and 
Basic Skills (JOBS), and the state-funded State Training and Employment Program 
(STEP). Individual institutions/programs require UI wage record information from the 
state agency. 

During the last several years, the Alaska Department of Labor has collected 
occupation and place-of-work information for most of Alaskans wage and salary 
employees. This unique data source called the Occupational Data Base (ODB) originated as 
a means of identifying the economic impact of nonresident workers. While the issue of 
residency is no longer the prime concern, Alaska continues to collect this data in addition to 
the federally required information in the UI wage record database. 

Alaska currently crossmatches the ODB which includes a four-digit occupational 
code and geographic location of employment and wage records with unemployment 
insurance data, Alaska Permanent Fund Dividend data, and other related information. In 
addition, it is possible to obtain information from other Pacific Northwest states about the 
industry and earnings of Alaska graduates that have left the state. However, studies of 
interstate mobility of vocational education graduates have not been attempted. 

Over the past few years, human resource development programs have begun to tap 
this database for employment and earnings status studies of their graduates, and tiie 
Department of Labor is actively mariceting their potential to various training entities. The 
Alaska Vocational Technical Center requested tiie Alaska Department of Labor^s analysis of 
its 1988 graduates' eammgs performance before and after training. Based upon its study 
"Employment Status and Earnings of Alaska Vocational Technical Center Training Program 
Graduates Completing Training in Fiscal Year 1988," tiie Alaska Department of Labor 
(1990, see Appendix D) was able to conclude tiiat "witiiout exception, when comparing 
earnings by course, training had a significant positive impact on graduates total and average 
earnings" (p. 2). 
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One year later, the Department of Labor analyzed the earnings performance of 
AVTECH 1989 graduates before and after training. Among its findings. 



• Nearly half (48%) of the 204 graduates working in wage and salary employment in 
Alaska the year following training were employed in industries and/or occupations 
which were related to their course of study. 

• Total earnings for all graduates were up 1 15% from the year before training began. 

Another study looked at program completers from the Martin Luther King Career 
Center, a secondary vocational-technical school in Anchorage, Alaska. The 259 students in 
the King Career Center study completed three or more credits in an occupational cluster in 
1987. Major findings included the following: 

• Overall, twenty-five percent of the completers working in wage and salary 
employment in Alaska in 1990 were employed in industries and/or occupations 
related to their course of study. 

• Average 1990 earnings of the completers exceeded those of other Alaska workers 
their age, both overall and by gender. 

Cooperative agreements have also been reached with other Alaska state agencies to 
use Department of Labor data sources to help evaluate the JOBS and STEP programs. 
STEP is an Alaska program funded by employee contributions to the Unemployment 
Insurance Trust Fund. 
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Table 3 
Alaska Profile 



Authorizatioa/Radonale 


In 1987, the Alaska Department of Labor 
supplemented the UI wage record data with the 
Occupational Data Base to assess the economic 
impact of nonresident workers. 


State Agencies 


Alaska Department of Labor 


Specific Data Files Used 


• UI wage record data 

* ODB (includes 4-digit occupational code and 
geographic area of employment) 


Vocational Education Target 
Group/Size 


Two studies have been done for local vocational 
schools— one secondary using 259 completers from 
1987 and one postsecondary with 204 graduates 
from 1989 


Demographic Data 


Age, ethnicity, gender 


Educational Data 


Occupational education course title or cluster, dates 
of training, geographic region of training 


Employment Data 


Occupational code (4-digit), industry of firm, 
quarterly wages, average earnings (one and three 
years), location of employment 


Analyses Conducted 


Reports are done for individual schools and 
institutes — comparisons of earnings of secondary 
vocational education concentrators with overall age 
group cohorts and pre- and pos^rogram earnings 
comparisons for postsecondary vocational*techmcal 
education completers. 


Major Findings 


• In the secondary school study, 25% of completers 
were woridng in occupations related to training and 
their earnings exceeded the earnings of their age 
cohorts who did not receive vocational education. 

• In the technical mstitute study, 48% of the 
completers were employed in training-related 
occupations and their posq)rogram earnings were 
115% of preprogram earnings. 


Format of Report 


Individual reports are prepared for schools and 
technical institutes and are available from the Alaska 
Department of Labor. 


Future Directions 


Cooperative agreements have been reached with other 
state agencies to use the UI and ODB data to evaluate 
JOBS and STEP (state-funded). 




Colorado 

Colorado law requires accountability of its postsecondary institutions by reporting 
"evidence of c^er-graduation performance. " Subsequent UI wage records matching data is 
shared with the postsecondary institutions for their use in program review and 
accountability. 

Amid the growing demands for educational accountability nationwide, the Colorado 
General Assembly in 1985 passed House Bill No, 1 187 which among other things required 
the following: 

That institutions of higher education be held accoimtable for demonstrable 
improvements in student knowledge, capacities, and skills between entrance 
and graduation; . . . and that institutions of higher education shall design 
and implement a systematic program to assess the knowledge, capacities, 
and skills developed by students in academic and cocurricular programs. 
(Article 13, Section 23-13-101(l)(a), (3)(a)) 

In Colorado, one area of accountability reporting is "evidence of after-graduation 
performance" in such areas as employment and continuing education. Under the leadership 
of the Colorado Community College and Occupational Education System (CCCOES), the 
state began to explore ways to generate a. ountability data at the state level and pass it back 
down to individual community colleges. By combining data provided by Colorado public 
postsecondary institutions with data available from the Colorado Department of Labor and 
Employment, the CCCOES has developed a longitudinal database which monitors 
students* postgraduation employment and continuing education behavior in what the state 
considers a cost-efficient maimer. 

The longitudinal student tracking system contains data from the following three 
sources: (1) the Colorado Commission of Higher Education (CCHE), (2) the Colorado 
Department of Labor and Employment (CDLE), and (3) the CCCOES. In 1989, the CCHE 
upgraded its Student Unit Record Data System (SURDS) to include an undergraduate 
applicant file, a student Fall enrollment file, a degree granted file, and a financial aid file. 

The CCCOES has been tracking after-graduation employment of its degree and 
certificate recipients through the UI wage files for a number of years. In general, the 
process proceeds as follows: 

1 . CCHE sends to the CCCOES a tape of recent graduate social security numbers. 



O 30 4^ 

ERIC 



2. CCCOES sends this tape to the Colorado Department of Local Affairs (CDLA). 

3 . CDLE send the UI wage record file to CDLA. 

4. CDLA crosswalks the CCHE and CDLE files and sends CCCOES a copy of 'iie 
matched file. 

5. CCCOES compares this data and matches it with previous years* data and with 
other administrative databases such as CCHE's SURDS. 

6. ;CCOES analyzes the data and forwards reports to individual community colleges 
for use in program review and accountability reports. 

The ir' ' 1 run was conducted on the 1985-1986 community college degree class 
because th^s he first group for whom data was available from all three sources. Of the 
3,797 degree and certificate recipients in 1985-1986, 19.6% were enrolled in additional 
educ n programs, and about 57.9% were found in the UI file approximately one year 
after graduation. Of these, 12.4% were both working and going to school. Approximately 
oiiv, .hird of the graduates were not identified in either file. One year later, a few additional 
graduates were identified in the UI file as working in Colorado. 

Colorado further identifies the crossmatched data based upon a number of variables 
such as college type attended, degree type received, gender, age, and ethnicity and can 
compare work status and an earnings profile across such variables. 
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Table 4 
Colorado Proflle 



Authorization/Rationale 


In response to new state mandates for increased 
educational accountability, the Colorado Community 
College and Occupational Education System 
developed a longitudinal database to monitor 
postgraduation employment and continuing 
education. 


Cooperating Agencies 


• Colorado Commission on Higher Education 

• Colorado Department of Labor and Employment 

• Colorado Department of Lxx:al Affairs 


Specific Data Files Used 


• UI wage records 

• Higher education enrollment 


Vocational Education Target 
Group/Size 


Initial study focused on 3,797 associate degree and 
certificate recipients from 1985-1986 


Demographic Data 


Gender, age, and ethnicity 


Educational Data 


Type of college attended, degree received, higher 
education status 


Employment Data 


Work status, earnings profile 


Analyses Conducted 


Data tables are developed by CCCOES and 
forwarded to each postsecondary institution for use 
in program review and accountability reports. 


Major Findings 


For the 1985-1986 graduating class: 19% were 
enrolled in higher education, 58% were in the UI 
wage record file (i,e„ employed) one year after 
graduation, and 12% of those employed were both 
working and going to school 


Format of Report 


Research report prepared by a state agency official 
for a regional conference. 


Future Directions 


Reports are provided to local community colleges 
upon request 
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Feasibility or Exploratory Work Completed/Planning Underway 

for Systemic Use 

Both Indiana and Illinois have studied using UI wage record information for 
follow-up purposes and both have decided to move forward — ^Indiana based upon its 
successful pilot study and Illinois based upon its successful JTPA utilization of the UI 
wage record data. 

Indiana 

The Indiana Commission on Vocational and Technical Education has identified a 
need for more accurate and timely data and information for planning, evaluation, and 
decision making. Indiana has conducted pilot testing with UI wage records and is ready to 
proceed. A reorganization in state government has occurred which combines vocational 
education, worlq)lace literacy, employment and training, UI, and JTPA into one department 
which sliould improve working relationships and communications. 

Since early in its existence, the Indiana Commission on Vocational and Technical 
Education (ICVTE) has acknowledged a greater need for accurate and timely information 
for planning, evaluation, and decision making. As in most states, the vocational education 
establishment in Indiana conducts an extensive follow -up of vocational program completers 
and leavers, utilizing traditional mail follow-up techniques. In 1989, the ICVTE began 
discussions of the use of social security numbers for matches with such databases as 
Unemployment Insurance, the Department of Defense for military personnel, and the 
Indiana Commission for Higher Education for those students enrolled in postsecondary 
education. It also required that as of the Fall of 1989, social security numbers were 
required as part of the enrollment process. 

During the Spring of 1990, the ICVTE completed a pilot project on fifteen-hundred 
student completers from the school year 1988-1989 using the three databases described 
above. Over seventy-one percent of the fifteen-hundred student completers were matched 
in one of three files with 66.5% employed, 3.4% in the military, and 16.7% enrolled in 
some form of postsecondary education. This seventy-one percent figure compares with the 
twenty-eight to twenty-nine percent which the state of Indiana previously found through its 
traditional mail follow-up. Plans are underway to examine all high school and adult 
completers from 1989-1990 in the near future. It is anticipated that soon all postsecon '-- y 




completers will be included in the system. Indiana closely follows the system developed in 
Florida (described earlier in this report) and currently crossmatches student files with those 
of UI, the Department of Defense, and the Indiana Commission for Higher Education. 

Lastly, it should be pointed out that a reorganization in state government has 
recently taken place, resulting in the establishment of a Department of Workforce 
Development which includes individuals formerly working in vocational education, 
workplace literacy, employment and training, UI, and JTPA programs. This 
reorganization should go a long way toward improving working relationships among all 
those involved. 
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Table 5 
Indiana Profile 



Authoiizadon/Rationale 


The Indiana Commission on Vocational and 
Technical Education has sought accurate and timely 
information for plaiming, evaluation, and decision 
making. It has and continues to conduct pilot 
studies. 


Cooperating Agencies 


• ICVTE 

• U.S. Department of Defense 

• Indiana Department of Employment and Training 
Services/Unemployment Insurance (UI) 

• Indiana Commission for Higher Education 


Specific Data Files Used 


Social security numbers matched for completers from 
five secondary vocational-technical education districts 
with active military personnel, enrollment in public 
postsecondary institutions, and UI wage records. 


Vocational Education Target 
Group/Size 


1988-1989 completers from five secondary 
vocational education programs/n=l,497 


Demographic Data 


None 


Educational Data 


Occupational program title 


Employment Data 


Employment (3rd and 4th quarter of 1989), military 
enlistment (as of September 1989), and enrollment in 
higher education (in 1989-1990) 


Analyses Conducted 


Sample aggregate reports were prepared for each of 
the five area secondary programs. For each of the 
matched employees, the ICVTE contacted employers 
for occupational titles and the county of employment 


Major Fmdmgs 


Overall, 71% of the completers in tl^ initial pilot 
were found in one of the three databases. 
Specifically, 66% were found in the wage records, 
16% were in higher education, and 3% were in the 
military. 


Format of Report 


Memoranda prepared by staff members for ICVTE. 


Future Directions 


The recent establishment of the Indiana Department 
of Woiiforce Development will most likely call for 
more interagency evaluation and accountability 
initiatives such as this. 
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Illinois 

The Illinois Department of Employment Security (IDES) has established an 
Employment Tracking System (ETS), a longitudinal database of seven years of information 
on wages, UI, and job sen^ice contacts, in order to provide various agencies access to the 
information. The department has also developed formed shared data agreements to facilitate 
that access. Illinois will use UI wage record information to comply, in part, with the 
requirements of performance standards under the Perkins Act 

The Illinois State Board of Education will be using the IDES-established ETS in 
monitoring its Perkins vocational education performance standards in 1992. ETS will be 
used to collect employment and earnings information on students served in vocational 
education at the secondary and postsecondary levels. 

The goal of the ETS is to provide various agencies, including the Board of Higher 
Education, the Community College Board, the Department of Aging, and the Department of 
Children and Family Services, access to information which evaluates the impact of their 
education or training programs on the persons they have trained. The JTPA system in 
Illinois has used the ETS measures in its performance system. Individual client 
information is available, however, only if the person continues to live and/or work in 
Illinois. 

An agency authorized to participate in ETS submits a tape containing the social 
security numbers (SSNs) of persons they wish to track to IDES. The SSNs are then 
entered into ETS and a tape containing the historical wage data is returned to the agency. 
Agencies seeking to utilize the information base of ETS enter into a formal Shared Data 
Agreement which, among other things, promises to maintain strict confidentiality of any 
data that is provided as well as describing costs to be reimbursed to IDES by the 
participating agencies. All development costs and all costs for the first year of operation 
were absorbed by IDES. A fee is charged per SSN tracked. 

A significant concern to Illinois are the metro labor markets it shares with Missouri, 
Iowa, and Indiana. Illinois is attempting to develop data exchange agreements with these 
states as well as to conduct matches of JTPA clients with their wage files. In addition, 
factors such as demographics, JTPA program intervention, and local economic conditions 
are to be addressed by use of multiple-regression statistical modeling. 
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In 1989, the Illinois Education and Employment Subcabinet began work on a 
Master Plan for Employment and Training. The Master Plan identified and reviewed 
current workforce preparation programs and proposed strategies for improved 
coordination. For the first time, the plan required a combined evaluation of state programs 
using ETS, which was developed by IDES. In 1990, the Education and Employment 
Subcabinet organized an Interagency Task Force led by the IDES to conduct a pilot study in 
using the ETS system to conduct a combined evaluation of state employment and training 
programs. The study was designed to develop a comprehensive set of performance 
measures and explore the problems and limitations in conducting combined program 
evaluation with the ETS system across nine state agencies. Although the final report of the 
pilot study has not yet been released, some preliminary observations have been offered. 
Program performance was to be evaluated in following three major areas: 

1 . Employment Measures. The employment measures utilized are employment rates 
representing the percentage of the former program participants who were employed 
in Ul-covered employment during a given postprogram quarter. 

2. Employment Retention Measures. The measures assess the degree to which 
participants remain employed over a specified period of time. 

3. Earnings Measures. The earnings measures attempt to assess the quality of 
employment, first by simply measuring the average earnings of participants during 
the first postprogram quarter of employment and then by extending that 
measurement over a given period of time. 

The tentative plans for the Vocational Education System of Performance Measures 
and Standards call for using ETS to assess the state goal which focuses on enabling 
students to "obtain employment which provides for economic self-sufficiency and career 
advancement." ETS measures for this goal will include employment rate, employment 
retention rate, average quarterly earnings, and training-related placement rate. To determine 
"training-related placement," ETS will have to be modified to include the collection of 
occupational data. Statewide standards for each measure are to be established in 1992- 
1993. The ETS employment data and analyses will be disaggregated to examine labor 
market and employment outcomes for special groups of former vocational education 
students (e.g., individuals who are disabled and others). 
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Table 6 
Illinois Profile 



Authorization/Rationale 


In 1989, the Illinois Department of Employment 
Security established a longitudinal employment 
tracking svstem (ETS^. which contains wace 
information, unemployment data, and Job Service 
contact information. In 1990, the Governor's 
Educational and Employment Subcabinet authorized a 
pilot study to conduct a combined evaluation of the 
state's various employment and training programs, 
including those overseen by the State Board of 
Education and the Community College Board. The 
ETS will be used in the Perkins System of 
Performance Measures and Standards for vocational 
education. 


Cooperating Agencies 


• IDES 

• ICCB 

• Nine other state agencies 


Specific Data Files Used 


• ETS data files on unemployment and wages for 
1989-1990 

• Social security numbers of completers 


Vocational Education Target 
Group/Size 


Occupational program completers in community 
college programs for 1988-1989/n=15,485 


Demographic Data 


None 


Educational Data 


Completion of an occupational education program 


EriploymentData 


• Employment during four quarters following 
graduation 

• Retention in employment for consecutive quarters 

• Quarterly earnings 


Analyses Conducted 


• Percent employed in quarters 1-4 following 
graduation 

• Percent remaining employed for two, three, or four 
quarters 

• Average earnings for those graduates employed 
each quarter 


Major Findings 


70% of the graduates were employed; retention in 
employment in subsequent quarters declined from 
91% (Qtr. 2) to 82% (Qu-. 4); average quarterly 
earnings mcreased from $4207 to $4621 during the 
first two quarters following graduation. 


Format of Report 


Technical Report of ETS multi-agency pilot study 


Future Directions 


ISBE win be using the ETS database to monitor 
employment and earnings of vocational education 
graduates as part of the Vocational Education System 
of Performance Measures and Standards. 
Consideration is being given to requesting the 
inclusion of occupational title in the database. 
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Feasibility or Exploratory Work Completed/ 
Future Development Or Use Uncertain 



A number of states completed single analyses or studies in utilizing the UI wage 
record, but for a variety of reasons, have not proceeded to develop their implementation 
processes more fully. Arizona completed a pilot study to provide consumers data on 
placement rates, but has run into problems over confidentiality of social security numbers. 
Missouri conducted a longitudinal follow-up of its high school graduates and used UI wage 
record data to corroborate the self-reported data from its graduates. New Hampshire's 
Department of Economic Security studied the employment and educational experiences of 
students while in high school. Oklahoma performed a feasibility study matching student 
records to wage reports and hopes to conduct additional, in-depth studies in the future. 
Texas has performed a number of studies and looks forward to securing additional 
resources to more fully develop its system. Virginia conducted a pilot study based upon 
community college instructional program areas. Finally, Wisconsin conducted a research 
study utilizing both UI wage record information and income tax information and compared 
postsecondary completers and their earnings to a comparison group established by U.S. 
Census Bureau data. 

Arizona 

Arizona attempted to use UI wage records to "provide consumers of postsecondary 
vocational education" with placement rates and earnings of community colleges and 
selected private schools and met with limited success. It was unique in including some 
proprietary schools. 

In 1985, the Arizona Legislature required the state to "publish and distribute a 
report of the placement rates and average salaries earned by persons completing vocational 
programs in this state." Subsequently, the Center for Vocational Education at Northern 
Arizona University undertook the responsibility of planning the report, collecting the data, 
and publishing an initial report m 1986. 

As stated in its first report, the Center's purpose is to "provide consumers of post- 
secondary vocational education programs data regarding placement rates and earnings 
reported by employers to the Department of Economic Security associated with programs 
found in community colleges and private schools.** 
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Social security numbers of program completers of public and private postsecondary 
schools were compared with the imemployment compensation files of employees reported 
by all employers in Arizona^ In its first year, one-hundred percent of the public institutions 
but only forty-two private schools out of 107 (38%) private schools licensed by the state 
were included in the study. Those not participating either had less than twenty-five 
program completers or did not collect social security numbers for the completers. 

The foUovdng caveats were offered in the study: 

• Since each individual school participating in the study defined on its own a 
"vocational program," this lack of uniform program definition prohibits meaningful 
comparisons of program graduates among schools as the programs were of varying 
lengths and focused on a wide range of occupations. 

• A number of private schools did not collect social security numbers of students. 

• The database of the Department of Economic Security includes part-time and short- 
term employment as well as those employed full-time all year. There are also a 
number of workers excluded from the database. 

• A number of program completers left Arizona for employment, entered the military 
service, or did not enter the labor market and are not included in the study. 

Since the completion of the pilot study referred to above, the State of Arizona has 
run into some problems over the confidentiality of social security numbers. Persons 
advocating the use of social security numbers as student identification and hence available 
for use in such research projects have not yet persuaded the Arizona Legislature to pass 
enabling legislation. 
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Table 7 
Arizona Profile 



Authorization 


The Ari2X)na Legislature required state officials to 
publish and distribute a report of placement rates and 
average salaries earned by postsecondary vocational 
education completers. 


Cooperating Agencies 


Arizona Department of Employment Security 
Contracted to the Center for Vocational Education, 
Northern Arizona University 


Specific Data Files Used 


• UI wage record data 

• Social security numbers of students in public and 
private postsecondary institutions 


Vocational Education Target 
Group/Size 


All fifteen community colleges, two skill centers, and 
forty-two private schools providing postsecondary 
vocational-technical education provided social 
security numbers for completers during 1983-1984 


Demographic Data 


None 


Educational Data 


Institution, vocational-technical education program 
(independenfly defined), number of completers per 
program 


Employment Data 


1984-1985 earnings 


Analyses Conducted 


For each program within an institution — number of 
completers in Arizona, number and percentage with 
earnings, average earnings, and median earnings 


Major Findings 


The lack of uniform definitions for postsecondary 
vocational education programs in this study limits the 
utility of the findings. Several private schools do not 
collect social security numbers. 


Format of Report 


Published report for 1986 


Future Directions 


Additional consumer-oriented studies have not been 
undertaken. Substantial concern exists regarding the 
use of social security numbers for the evaluation of 
programs. Cunenfly there are no plans to continue 
additional development 
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Missouri 

As part of a longitudinal, five-year study of secondary school students, Missouri 
compared vocational education completers with nonvocational education completers and 
used the UI wage data to corroborate the self-reported information provided by the 
completers. 

The Missouri Department of Elementary and Secondary Education in cooperation 
with the Missouri Testing and Evaluation Service, University of Missouri at Columbia, 
undertook a longitudinal study of a sample of Missouri graduates from the class of 1981 
which was extended over a five-year period. The overall purpose of the study was to 
determine the successes or difficulties high school graduates faced in the transition from 
high school to inmiediate post-high school years. 

Thirteen area schools providing secondary vocational-technical programs 
participated. A random sampling procedure was employed to produce roughly equivalent 
samples, one consisting of vocational graduates and a second consisting of nonvocational 
graduates. State definitions were utilized to differentiate the two. The study was designed 
to compare and contrast the samples of vocational education and nonvocational education 
students, focusing primarily on economic aspects relative to salary and employment 

In the initial phase of the study, traditional follow-up techniques were utilized. 
Within the design of the study, former graduates were classified into vocational education 
completers and nonvocational education completers. During the five-year study, former 
students were provided personalized letters, and personal, subjective comments on the 
post-school experiences of students were actively solicited. Lastly, employment data and 
unemployment claims information from the Missouri Department of Employment Security 
(DES) were utilized to corroborate the self-reported data provided by the graduates. 

In this latter phase of the study, wage record data was used. From the study's 
original sample of 1,514 students, 1,400 social security numbers were matched. Of these, 
705 were from the sample of vocational education graduates and 695 were from the 
nonvocational students. These social security numbers of both groups were matched with 
data provided by DES which included 

o 

• applications for services provided by DES, 
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• Unemployment Insxirance claims information, 

• selected quarterly wages, and 

• number of quarters worked since graduation. 

Among the conclusions drawn for the report based upon the analysis of DES data 
are the following: 

• More vocational than nonvocational graduates applied for services provided by DES 
(e.g., counseling, job referrals, and employment testing) from the 1981 through 
1986. 

• From the second quarter of 1981 through the second quarter of 1986, fifty-nine 
percent of the vocational graduates and sixty-five percent of the nonvocational 
graduates were actually paid unemployment compensation, 

• The vocational education graduates showed consistently higher average fourth 

quarter wages from 1981 through 1985 than the nonvocational graduates in aU three 

of the categories — full-time work, part-time work, and other/unknown status (e.g., 

military, wage record data, but no returned questionnaire). 
• 

• The number of quarters worked from the second quarter of 1981 through the first 
quarter of 1986 indicated that vocational education graduates were working a 
median of thirteen quarters, while nonvocational graduates worked a median of 
eleven quarters. 
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Table 8 
Missouri Profile 



Authorization/Rationale 


UI wage records were used in a five-year 
longitudinal study of 1981 high school graduates 
sponsored by the Vocational and Adult Education 
Section of the Missouri Department of Elementary 
and Secondary Education* 


Cooperating Agencies 


• University of Missouri at Columbia 

• Missouri Department of Employment Security 


Specific Data Files Used 


• DES administrative files (UI wage records) 

• Social security numbers of 1981 graduates 


Vocational Education Target 
Group/Size 


1981 graduates of secondary vocational-technical 
education centers/n=705 


Demographic Data 


None reported 


Educational Data 


Completion of a secondary vocational-technical 
education program 


Employment Data 


• Services received from DES 

• UI insurance claims information 

• Waces for selected ouarters 

• Number of quarters worked since graduation 


Analyses Conducted 


Comparisons of employment data for graduates of 
vocational education nro crams and graduates of 
nonvocational education programs. 


Major Findings 


Graduates of vocational education programs showed 
consistentiy higher average fourth quarter earnings 
from 1981-1985 compared to nonvocational 
graduates. Vocational education graduates were 
employed more consistently than nonvocational 
education graduates. 


Format of Report 


Two research reports are available, including a 
summary report on the longitudinal study and a 
special report on the analysis of the DES data. 


Future Directions 


None indicated in the reports. 
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New Hampshire 

New Hampshire investigated the labor market activities and experiences of high 
school aged youth and looked at five different cohorts who were tracked and compared 
over a four-year period 

Although not specifically directed toward utilizing wage record data in the 
assessment of vocational education graduates or completers, the state of New Hampshire 
has recently completed an interesting study focusing an analysis of teenage labor market 
behavior and the levels of educational attainment. In the latter 1980s, New Hampshire 
experienced relatively good economic prosperity and a resultant tight labor market This 
caused salaries to increase and attract high school aged youth — ^who might not otherwise — 
to work to a greater degree. Problems arose with a somewhat rising rate of dropouts. A 
legislative committee attempted to study the problem, but both educational and employment 
data were inadequate. The Commissioner of the Department of Employment Security 
decided to investigate the labor market activities of these high school aged youth. 
"Working & Schooling Decisions: A Study of New Hampshire Teenage Labor Market 
Behavior and the Level of Education Attainment" (see Appendix D) attempts to answer the 
following questions among others: 

What types of jobs are noncollege-bound youth qualified for after leaving 
high school? Ho' ^ well do high schools prepare noncollege-bound youth 
for the workplace? 

What industries hire in-school teenagers and to what extent do these 
industrial categories change once students leave school? 

What is the relationship between educational outcome measures and 
productivity on the job? Do improvements in the former translate into 
improvements in the latter? 

What institutional arrangements currently exist to aid youth in the school-to- 
work transition process and what changes need to be made to smooth 
transition paths? (p. 1) 

Eighth- through twelfth-grade cohorts were established and data was tracked by 
sex, region of the state, and program type (i.e., academic, vocational, or general tracks). A 
total of 1,220 parental/guardian consent forms were obtained for students to participate in 
the study. Of these, 1,140 matches were found in the New Hampshire Unemployment 
Insurance Quarteriy Employment and Wage Records or about 93.5% of those consenting to 
be involved in the study. 
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The New Hampshire study provides new strategies for the utilization of wage 
record data for tracking students still enrolled in school and their experiences in the labor 
market. This has implications for how vocational educators, particularly on the 
postsecondary and adult levels, may wish to use wage record data in a follow-up situation. 
Data may be collected from years prior to enrollment, during enrollment, and then a number 
of years following course or program completion to construct an accurate earnings or work 
profile of vocational education completers. 

The foUov/ing three recommendations emanating from the New Hampshire report 
merit consideration: 

1. Open new and expand old lines of communication between high schools and 
employers to foster tlie schooi-to-work connection. 

2. The opportunities and alternatives av/dlable to noncollege-bound youth should be 
expanded. 

3. The State Departments of Employment Security, Education, and Postsecondary 
Technical Education, working in conjunction with the State Occupational 
Information Coordinating Committee (SOICC), the University System of New 
Hampshire, and the New Hampshire Job Training Council should design a 
database integrating the employment, training, and educational histories of New 
Hampshire youth. This comprehensive integrated database would allow further 
tracking of youth beyond high school, thereby providing additional information on 
the school-to-work transition process. It can also serve as a valuable assessment 
tool. At the heart of this database would be the Unemployment Insurance Wage 
Credit Records. 
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Table 9 
New Hampshire Profile 



Authorization/Rationale 


Beginning in 1987, the New Hampshire Department 
of Employment Security launched the New 
Hampshire Teenage Labor Maxkti Study to examine 
the employment and education experiences of youth 
during their high school years. 


Cooperating Agencies 


New Hampshire DES 


Specific Data Files Used 


• UI wage record data 

• High school transcripts 


Vocational Education Target 
Group/Size 


Five grade-level cohorts (grades 8-12) were followed 
from four years (1987-1990)/n=l,220. This was a 
statewide representative sample of the population of 
students in grades 8-12. Of the students in the 
sample, 107 were in the vocational education track. 


Etemographic Data 


• Gender 

• Region of high school in the state (3 regions) 


Educational Data 


Transcripts provided by high schools indicated the 
track of each student included in the sample 
(vocational, general, academic, and unknown). 


Employment Data 


For 1,140 usable wage record files — employment by 
quarter (1987-1990), quarterly hours worked, annual 
and quarterly earnings, and standard industry 
classification (two-digit). 


Analyses Conducted 


The employment data was analyzed for each of the 
five cohorts over the four-year period ^^ith attention 
to the high school program tracks. 


Major Findings 


99% of the 1989 seniors were formally employed at 
some time during 1987-1990; 71% worked in at least 
three quarters. Students in a vocational or general 
education program worked significantly more hours 
per we^k than academic track students, primarily in 
retail trade industries. The 1990 economic downturn 
in New Hampshire has had a profound effect on 
youth employment which may affect future 
employment and income stability. 


Format of Report 


Series of three research reports published by the New 
Hampshire DES (see Yasuda, 1991 [see Appendix 
D]). "^^^ 


Future Directions 


A major recommendation in the report suggests that 
several state departments (education, postsecondary, 
technical education, and employment security), the 
university and the SOICC design and use a database 
integrating the employment and education histories of 
New Hampshire youth, using UI wage records as the 
core. 
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Oklahoma 

The Oklahoma Department of Education conducted a feasibility study that showed 
matching student records with wage records to be an effective method for assessing 
placement of vocational education completers. Oklahoma also included a sizable number of 
adult vocational education completers in the study. 

The Oklahoma Department of Education undertook a study in 1989 to determine the 
feasibility of matching student records to employer wage reports to assess vocational 
completer labor market outcomes. The only aspect of labor market outcomes to be 
investigated was job placement since the primary purpose of the study was to examine the 
methodology. 

The department provided student completer data consisting of 14,907 records 
(social security numbers and CIP code) from the 1986-1987 school year to the Oklahoma 
Employment Security Commission (ESC) for matching. Of these completers, 73.1% were 
secondary vocational education completers while the balance were adult completers. For 
purposes of this study, if an individual was working in one, part, or in all four quarters of 
1988, they were counted as a match. 

Of the 14,907 completers submitted to the ESC, 10,340 (69.4%) wage record 
matches were obtained for 1988. Secondary completers had a match rate of 69.1% while 
adult completers were matched at 70.1% Analyses were also performed to ascertain the 
relationship between the CIP and the standard industrial classification (SIC) of the 
completers* employers. 

Some summary statistics from the report are of interest: 

• The percentage of secondary completers working in any given quarter varied as 
much as 11% across four quarters of 1988. The percentage of adult completers 
working each quarter was greater than 80% and remained stable between quarters. 

• The percentage of secondary completers who woiked each quarter but m a different 
industry was 47.2%. Sixty-three percent of adult completers worked in at least two 
different industries during 1988. 
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• The number of adult completers who worked in the same industry throughout 1988 
was nearly double (40.5%) that of secondary completers who did so (22.6%). 

• There were ten standard industrial classifications (SIC)s in which 100 or more 
completers were working at some time in 1988. These ten SICs employed 42.2% 
of all vocational completers who worked in 1988. 

But perhaps of more interest are some of the conclusions about the methodology of 
using wage record data that the Oklahoma study put forth: 

• Matching student records against employer wage records has been shown to be an 
effective method for assessing industry placement of vocational completers. 

• Further in-depth analyses is needed to examine some of the limitations inherent in 
the methodology such as the possibility of collecting occupational titles, at least at 
the time of initial employment 

• Other databases need to be looked at, at both the state and federal levels, to identify 
the approximately 30% of completers not found in the wage record fUes. 

• In most cases, the majority of vocational completers in a given occupational field 
are not concentrated in selected SICs but are finding employment in as many as 259 
SICs. However, thirty-six SICs employed 64.9% of the 1986-1987 completers. 
Some interpretations advanced for this phenomenon include (1) broad staffing 
patterns in any given SIC (in the oil and gas service industries, completers could be 
employed as auto service technicians, accountants, and secretaries) and (2) broad 
training in any given occupational program (graduates of accounting programs 
could be using their skills in several industries, e.g., department stores, auto 
dealerships). 

• Placement is a single dimension of labor market outcomes, which in themselves are 
a single indicator of vocational education effectiveness. When assessing the 
effectiveness of vocational education, it is extremely important to consider ''I) 
multiple outcome measures and (2) year-to-year improvement measures. 

Although this was a one-time study for the Department of Vocational and Technical 
Education, it is their hope to conduct additional and more in-depth studies using this 
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methodology as soon as new workmg relationships can be established with the Oklahoma 
Employment Secxirity Commission. 



Table 10 
Oklahoma Profile 



Authorization/Rationale 


The Oklahoma Department of Vocational and 
Techmcal Education sought to examme the feasibihty 
of matching student records to employer wage 
records to assess vocational completer labor market 
outcomes. 


Cooperating Agencies 


« Oklahoma Department of Vocational and Technical 
Education 

• CMdahoma Employment Security Commission 


Specific Data Files Used 


• 1986-1987 state generated vocational completer list 

• 1988 hourly and wage records of the Oklahoma 
Employment Security Commission 


Vocational Education Target 
Group/Size 


1986-1987 vocational education completers/ 
n=14,907: secondary, 10,888 or 73.1%; and adult, 
4,019 or 26.9% 


Demographic Data 


None 


Educational Data 


Vocational-technical education program (CIP code) 


JCLIIipiUyuiCIll UdXai 


employee cunng one or more quarters m lyoo and 
the SIC code 


Analyses Conducted 


Number of completers who worked in 1988, number 
of completers who worked in the same industry in 
each quarter, and employment of completers in 
specific SICs overall and by specific occupational 
division. 


Major Findings 


70% of the completers worked at some time in 1988. 
The percentage of adult completers who maintained 
stable employment and same-industry employment in 
1988 was higher than for secondary completers. Ten 
SICs employed 42% of all completers. 


Format of Report 


Research report available from the CMahoma 
Department of Vocational and Technical Education. 


Future Directions 


None indicated. 
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Texas 

Texas conducted a feasibility study which concluded that the expected benefits of 
UI wage reporting for follow-up purposes would reduce collection costs, facilitate 
standardized collection, provide for integrated data analysis, and provide data to regional 
and local planning processes. 

The State of Texas recently conducted a number of studies attempting to examine 
the feasibility of using UI wage records to demonstrate the outcome of public education and 
training programs. Across state government, the State Occupational Information 
Coordinating Committee (SOICC) under contract with Arthur Anderson and Company, 
studied the outcomes of state-funded education, training, and employment program 
participants. The study culminated in a definition of the statewide database of 
occupationally specific data within a proposed Quarterly Employment Tracking System 
(QETS). The pilot concluded that utilizing QETS was a viable approach to meeting 
accountability standards in Texas, but it also identified a need to enhance the current wage 
record reporting system so that such information as occupational and wage identifiers 
would be included. This would require legislative action and access to additional 
resources — resources that, as in most states, do not appear to be available at the present 
time. Currendy, full implementation of QETS is on hold. 

Potential benefits identified in the SOICC/Arthur Anderson and Company report if 
fully implemented follow: * 

• Reducing collection costs to the State of Texas by consolidating data 
collection/reporting functions through the use of existing databases and systems. 

• Facilitating the standardized collection of follow-up data for education, training, and 
employment programs m Texas and, thus, improving the quality of the data. 

• Providing an available source for integrated data analysis and follow-up for the 
Texas Human Resource Investment System. 

• Reducing the amount of staff time and postage costs associated with locating former 
program participants and their respective employers to conduct follow-up surveys. 

• Providing occupational placement and wage rate data to statewide regional and local 
planning processes. 
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More specific to the field of education generally and to vocational education in 
particular was a study undertaken by the Center for the Study of Human Resources at the 
LB J School of Public Affairs, University of Texas at Austin, under contract with the North 
Harris County College District in 1989 to design, develop, and pilot a postprogram follow- 
up system for postsecondary vocational education students in Texas. Originally, the 
project was to rely on a mail or telephone survey of program year 1988 students from four 
community colleges in Texas. However, it was later decided to conduct a follow-up pilot 
study of completers and nonretuming students using administrative records, most notably 
UI records from the Texas Employment Commission (TEC) whose database spanned a 
fifteen-month period and included quarterly earnings and the employer's name, financial 
reporting location/address, and industry. 

Of the 8,612 students mcluded in the pilot study, approximately eighty-five percent 
were found in the UI wage record database at least once during the five quarters under 
review. While usmg conventional, self-reporting techniques, only 34.9% of the vocational 
education completers were identified, while, in sharp contrast, 73.8% of the completers 
were located in covered employment in the third quarter of 1988 using the UI wage record 
files. 

Costs became a significant part of the study. All 8,612 student records were 
processed by the TEC at a cost of less than $86. Statistical analysis at TEC was 
approximately $2,500. More than 7,200 former students were found in the UI wage 
record file with a cost per student match of less than $0.36 and a cost of less than $0.08 for 
each wage record accessed. On the other hand, it was estimated that the conventional 
survey approach utilized by most postsecondary education institutions in Texas costs at 
least five to ten dollars per completed survey. 

The study concluded that "by matching student records with UI wage records an 
informational void can be readily filled at a very low cost" It further states that information 
obtained from UI wage records can also be used for the following: 

1. To demonstrate value-added competencies; 

2. To determine the various career paths taken by students in specific programs; 

3 . To track student mobility; and 
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4. To assess the durability and applicability of skills in the current labor market 
through retention studies, 

Finally, the study recommends that the Texas Higher Education Coordinating 
Board and two-year postsecondary institutions should consider adopting UI wage records 
as a permanent, follow-up data source as well as investigating the expansion of the sources 
of administrative records (i.e., adjoining state UI files, educational, military, and federal 
records). 




Table 11 
Texas Profile 



Authorization; Rationale 


Two major studies have been done in Texas to 
examine the feasibility of tising UI wage records to 
demonstrate the outcomes of public education and 
training programs. 


Cooperating Agencies 


• Texas Occupation Information Coordinating 
CcHnmittee 

• Texas Higher Education Coordinating Board 

• Texas Employment Conmiission (TEC) 


Specific Data Files Used 


• UI wage records 

• Completing and nonietuming student records from 
four postsecondary education institutions 


Vocational Education Target 
Group/Size 


Completers and nonietuming postsecondary 
vocational-technical education students in 1988 
program year from four institutions. Group 
size/n=8,162 


Demographic Data 


None specified 


Educational Data 


Program completion status (completer or 
nonretuming), instructional program completed/ 
started (CIP code) 


Employment Data 


Employment by industry (4-digit SIC code), median 
quarterly earning 


Analyses Conducted 


Comparisons of completers and nonretuming 
students on the following: percentage employed vs. 
unemployed, employed in the same or different 
industry over five quarters, and median earnings over 
five quarters 


Major Findings 


Approximately 85% of the former students were 
found in the in wage record data during the five 
quarters. Nonretuming students appeared to be more 
likely to be employed while attending school, which 
may have adverse effects on their ability to complete 
the programs. 


Format of Report 


Research report on the pilot study. Center for the 
Study of Human Resources, University of Texas at 
Austin 


Future Directions 


Citing significant cost savings over conventional mail 
or telephone follow-up studies, the report 
recommends that the Texas Higher Education 
Coordinating Board woik with the two-year 
institutions to adopt and expand the use of 
administrative records such as UI wage records and 
other data on a permanent basis. 
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Virginia 

Virginia's Employment Commission studies postsecondary vocational graduates 
from 1975, 1980, and 1983 using the Ulwage records. In the report, the graduates were 
clustered by various curricula. 

In August of 1988, the Virginia Employment Commission (VEC) published 
"Tracking Education Completers" which describes the development of the computer 
tracking system intended to provide employment and wage information on graduates of the 
Virginia Community College System (VCCS). Similar methods were also used for 
vocational graduates and participants of JTPA, but no reports were written for these 
projects. The information collected facilitates the evaluation of graduates, classes, program 
effectiveness, and return on human capital 

The completer tracking system merges the VCCS student record file and the VEC 
unemployment insurance wage and tax files. The VCCS student record file contained 
approximately 22,200 student records from the 1975, 1980, and 1985 graduating classes 
and contained individual student data such as social security number, graduation year, sex, 
race, year of birth, curriculum, degree, credits, campus and college code. The VEC files 
include information on ninety to ninety-five percent of the state's wage and salary 
employment The UI wage files include information in Standard industrial Code (SIC) and 
wages. 

The merged data file contained approximately fifteen-thousand students, 
representing sixty-eight percent of the education completers from the VCCS files. The 
report, "Tracking Education Completers," profiles the 1985 graduating class of which there 
were about 6,210 matched records. Demographics for the graduating class are presented 
by sex, race, and age. Graduates were then clustered by the following curricula: business 
technology, engineering and industrial technology, college transfer, health technology, 
public service technology, art and design technology, and agricultural and natural resources 
technology. 

Further breakdowns were performed within each curriculum cluster again by sex, 
race, and age and then by average total wages along with its standard deviation. Finally, 
analysis was accomplished by aggregating the employment of graduates by two-digit SICs 
and distributing them by curriculum cluster. 




The following are two of the conclusions drawn in the summary section of the 

report: 

1. The majority of the matched graduates were female. The only exception by 
curriculum area was engineering and industrial technology. 

2, Average total wages were highest for the engineering and industrial technology 
group. The public service technology group was ranked second. The remaining 
technology groups were as follows: health, business, college transfer, art and 
design, and agricultural and natural resources. 



Table 12 
Virginia Proflle 



AuthorizationyRationale 


Over the last three years, the Virginia Employment 
Commission has indicated to various state agencies 
ihe availability of the UI wage record data. Reports 
have been done for the high schools, community 
colleges, and JTPA programs. 


Cooperating Agencies 


• VEC 

• Virginia Community College System 


Specific Data Files Used 


• UI wage records 

• Social security numbers of community college 
graduates 


Vocational Education Target 
Group/Size 


1985 graduates of the Virginia community colleges/ 
n=6,210 matched records 


Demogmphic Data 


Gender, age, race 


Educational Data 


Instructional program, degree, credits, campus, 
college, and graduation year 


Employment Data 


Average annual wages (based on five quarters from 
Quarter 1 in 1986 to Quarter 1 in 1987) 


Analyses Conducted 


Earnings data was reported by the curriculum area of 
the graduates. 


Major Findings 


Engineering/industrial technology graduates had the 
highest earnings ($24,260) and agricultural graduates 
had the lowest ($16,071). Differences in salaries 
between male and female graduates were also 
reported. 


Format of Report 


Report published by VEC, August 1988. 


Future Directions 


Use of the UI wage records is being considered by 
the Perkins Committee of Practitioners. 
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Wisconsin 

Wisconsin utilized both UI wage record data and state income tax data to track 
postsecondary vocational education completers and developed a statistical comparison 
group using U.S. census data against which to compare the completers. In addition to the 
usual educational and demographic data employed in other studies, the Wisconsin study 
differentiated among completers by length of program^ by possession of an economic 
disadvantage, and by various regions within the state. 

In the spring of 1988, the Wisconsin Board of Vocational, Technical, and Adult 
Education (VTAE) contracted with the Vocational Studies Center at the University of 
Wisconsin at Madison to conduct a follow-up study of the postsecondary system's 
graduates by utilizing state income tax information. A total of 15,959 VTAE graduates 
were identified for the school year 1982-1983. In addition to the social security number for 
each of these graduates, other information obtained for each person included gender, age, 
race, program instructional area, length of instruction, district in which the instruction was 
obtained, and whether the student received financial aid while attending school. 

This information was matched against the Wisconsin Department of Revenue's files 
for tax returns filed in 1985. A total of 12,982 matches were obtained— 81.3% of 1982- 
1983 VTAE graduates filed income tax returns in the state of Wisconsin in 1985 or 
approximately two to three years following graduation. 

During the course of that investigation, it came to the researchers' attention that the 
state of Wisconsin began a Wage Reporting System, administered by the Department of 
Industry, Labor, and Human Relations (DILHR). This system became fully operational in 
the first quarter of 1988. The tape of 1982-1983 VTAE graduates was run against the first 
quarter of 1988 DILHR tape and 11,220 matches were found, indicating that approximately 
seventy percent of the 1982-1983 VTAE graduates earned income in the state of Wisconsin 
during the first three months of 1988— ahnost five years after graduation. 

Numerous comparisons were made between and among different groups of 
graduates based on the variables mentioned earlier for both income reported in 1985 and 
1988. Some of these comparisons follow: 




• The longer the term of training, the higher the earnings of the vocational education 
graduate. The only clear exception to this rule was individuals completing short- 
term trade and industry programs who in many instances out-perfonned some two- 
year program completers. 

• Graduates from technical programs (two years in length only) and trade and 
industry programs earned more than graduates from all other program areas. 

• There was no discernible difference in earnings between graduates who received 
student financial aid while attending school versus those who did not In other 
words, being economically disadvantaged while attending school had little effect on 
earnings in the workplace. 

• Postsecondary vocational education graduates received unemployment 
compensation less often and to a lesser extent than the general population in 
Wisconsin. In the first quarter of 1988, no 1982-1983 graduate received any form 
of unemployment compensation. 

The researchers utilized U.S. Bureau of the Census data to develop a contrast group 
against which the 1982-1983 VTAE graduates might be compared. Utilizing the document, 
"Money Income of Household, Families, and Persons in the United States: 1985," as a 
starting point, this data was adjusted to conform to the age and gender distribution of the 
1982-1983 graduate population. This data was further adjusted for Wisconsin using the 
1985 ratio of Wisconsin per capita. The adjustment factor for 1985 was 0.953783; in other 
words, Wisconsin workers on average earned a little over ninety-five percent of the 
national average. Where comparisons were made for the five geographical regions of 
Wisconsin included in the study, per capita income for the five regions was derived by 
computing a weighted average of individual county per capita incomes published in 
"Current Population Reports," Series P-26, No. 86-ENC-SC, U.S. Bureau of the Census. 
Again, numerous comparisons by program area and by length of instruction were made to 
the comparison group, some of which follow: 

1 . Overall, postsecondary vocational education graduates economically out-performed 
comparison groupings based on the same gender and age breakdown for persons 
with a high school diploma. 




2. 



The longer the graduate is in the workforce, the more that graduate economically 
out-performs his or her peers. 



3. For the most part, female graduates out-performed their peers more than males 
when looking at percentages, although actual dollar amount differences were greater 
for males in 1988. 

4. Compared to national statistics for 1985, the 1982-1983 graduates tend to work in 
year-round, full-time jobs to a greater degree than their peers with high school 
diplomas. 

Among the many observations and conclusions reached in the report is the 
following: "The design of the research proved to be appropriate and highly feasible for 
carrying out follow-up research on vocational-technical students. In addition to a follow- 
up of recent completers, the methodology can be used for longitudinal studies. The 
potential exists to follow-up completers of training programs throughout their work 
careers." 



Table 13 
Wisconsin Profile 



A nthnri ratinii/Rationalc 


Study commissioned by the Wisconsin Board of 
Vocational, Technical and Adult Education to the 
University of Wisconsin at Madison. 


Cooperating Agencies 


• Wisconsin Department of Industry, Labor, and 
Human Relations 

• Wisconsin Department of Revenue 


Specific Data Files Used 


• UI wage record files 

• State income tax data 

^ WB VTAE student record files 


Vocational Education Target 
Group/Size 


1982-1983 graduates of Wisconsin's sixteen 
teclmical colleges/n=15,959 




Gender, age, and region of the state 


Educational Data 


Instructional program area, length of instruction, and 
disadvantagement 


Employment Data 


• 1985 WiscoKsLn income tax data 

• 1988 Fall quarter wage record data 


Analyses v^onaucteo 


PiiminoQ nf nropmm comnleters two-and-a-half vears 
after graduation (income tax data) and five years after 
graduation (UI wage record information) were 
compared to expected cohort group established by 
census data, broken down by program, age, and 
gender. 


Major Findings 


• 81% of graduates found in income tax database 
(two-and-a-half years after graduation) 

• 70% of graduates earned income in Wisconsin 
during Fall 1988 quarter (five years after 
graduation) 


Future Directions 


Uncertain at this point in time 


Format of Report 


Published research report 



ERIC 



70 

60 



FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 



Initially, this section presents the major findings and conclusions derived from the 
recent experiences of thirteen states in developing and using UI wage records to assess 
vocational education programs and policy. In addition, a set of recommendations are 
provided for federal and state policymakers relative to the expanded use of UI wage 
records. These recommendations focus, in large part, on improving the accountability 
mechanisms for vocational-technical education in an era of changing focus for programs, 
particularly at the secondary level. Finally, this section suggests several possibilities for 
promising development and demonstration initiatives which would explore the feasibility 
and utility of linking various administrative databases to monitor former student or graduate 
performance in the labor market or postsecondary education. 

Findings 

The findings which follow address three critical aspects of the effort in thirteen 
states to develop and use UI wage records: the initial authorization rationale, the various 
data sets employed in the studies, and the analyses conducted. 

Authorization/Rationale and Cooperating Agencies 

The impetus for undertaking studies of UI wage record data and their relevance to 
evaluating vocational-technical education programs comes from several sources. As noted 
in Table 14, authorizing directives were most commonly in the form of pilot or feasibility 
studies and emanated from state legislatures, state boards of vocational education, and state 
departments of labor or economic security. These directives were issued in most states in 
the mid to late 1980s and, in most mstances, culminated in reports one to three years later. 
In a smaller number of states, there appears to be a growing interest in developing better 
consumer information on vocational-technical education programs by providing summary 
data on the labor market experiences of graduates. 

Clearly the most comprehensive and significant authorization can be found in 
Rorida where the legislature enacted a bill requiring the state department of education to use 
a job placement standard in funding vocational education programs and created the Florida 
Educational and Training Placement Information Program as the state's primary follow-up 
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data collection tool. A minimum job placement standard is to be met by those programs 
focusing on specific job preparation if they wish to continue to receive federal and state 
funding. 

More commonly, state directors of vocational education in several states (Indiana, 
Oklahoma, Texas, Washington, and Wisconsin) have funded feasibility studies to 
determine the adequacy ofUI wage record data in their states for evaluating programs and 
services. In some instances, social security numbers were collected and matched for 
statewide cohorts, while in other states the studies were done with a selected sample of 
schools or postsecondary institutions. Not surprisingly, more of these studies focused on 
postsecondary than secondary vocational education programs because of the wider 
availability of students* social security numbers. In Missouri, the state director funded an 
analysis of the UI wage record data in conjunction with a longitudinal study of the 1981 
high school graduates. 

In several states (New Hampshire, Virginia, Alaska, and Illinois), the state 
department of labor or employment security (i.e., the state agency responsible for UI wage 
record reporting) has sought to initiate expanded use of the database. Heads of these 
agencies viewed the UI database as an underutilized resource impacting public policy. In 
some instances, these agencies have worked directly with state boards of vocational 
education in planning and conducting studies, while in other states the department has 
conducted matching studies for selected schools or technical colleges. 

In Arizona and Minnesota, increased concern from state legislatures for consumer 
information created the studies. Legislators in these states expressed interest in providing 
future students and their parents with more definitive information about the labor market 
experiences of students who had completed certain types of postsecondary vocational- 
technical education programs. 

Since the collection and maintenance ofUI wage record data is handled by a state 
department of labor in all but two of the states, collaboration with educational institutions or 
state education agencies at some level is required in order to complete analyses of the labor 
market experiences of graduates or former students. Throughout the thirteen states, 
various configurations of collaboration occurred, based on the structure of state 
government and the purposes of the study (see Table 14). 
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SPECIFIC DATA FILES USED 1 


• Ulwageiecoixis 

• Completing and nonreturning student 
records finom four postsecondary 
education institutions 


• Ulwagerecords 

• Social security numbers of 
comm inity college graduates 


• Ulwagerecords (Washington, Alaska, 
California, Idaho, and Oregon) 

• Higher education student recofxls 


• UI wage recoid files 

• Slate income tax data 

• WBVTAE student recoid files 


CD 


14 (continued) 


COOPERATING AGENCIES 


• Texas Occupation ^formation 
Coordinating Committee 

• Texas ffigher Education Coofdinating 

• Texas Employment Commission 


• virgmiai^mpioyment Lomnussion 

• Virginia Community College System 


• Dqpartment (rf Employment Security, 
Office (rf Fmancial Management 


• Wisconsin Department of Industry, 
Labor, and Human Relations 
(WBVTAE) 

• Wisconsin Department of Revenue 




Table 


AUTHORIZATION/RATIONALE 


Two major studies have been done in Texas to examine 
tbe feasibility (rf using UI wage records to demonstrate 
the outcomes of public education and training programs. 


13 S i « 

2 ? 8 " 

1 1 

salll 


me wasnmgion uoaru ior community College 
Education sought to conduct a pilot study of the 
feasibility of using employment security and higher 
education files to track the emptoyment and ednration 
outcomes of vocational education program completers. 


Study commissioned by the Wisconsin Board of 
Vocational, Technical, and Adult Education to the 
University of Wisconsin at Madison. 






STATES 


Texas 


I 

S 


i 

t 


Wisconsin 
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In some states, the comprehensiveness of the study required collaboration from a 
number of agencies — ^for example, in Florida and Washington the UI wage records are 
only one source of data for examining the postprogram experiences of vocational education 
students. Accessing data from other sources such as higher education institutions and 
federal employers such as the Postal Service or the military, requires that these agencies 
provide employment, earnings, enrollment, and other types of information. In several of 
these studies, new state-level data sharing agreements have been established. 

Across the thirteen states included in this study, collaboration and data sharing exist 
between state agencies, between federal and state agencies in a few states, and between 
state agencies and local schools or postsecondary institutions. In some states (Florida and 
Washington in particular) there are indications that data sharing by state agencies in 
adjoining states is beginning to be seriously considered. Multistaic data sharing will enable 
states to track the mobility of graduates or former students. 

In four states (Arizona, Missouri, Texas, and Wisconsin), the actual studies were 
conducted under the auspices of a grant or contract wi^ a university-based research group. 
These approaches were used as a matter of convenience or efficiency, since the state 
agencies or state boards lacked the specialized expertise, stc^time, or resources to conduct 
the studies internally. 



Data Sets — Demographic, Employment, and Educational Information 

Table 15 identifies the specific data files that were matched in each of the thirteen 
states conducting studies. These data files contained a variety of demographic, educational, 
and employment data on a target group of former vocational education students. 

In these studies, the target groups ranged from two-hundred secondary vocational 
education graduates of a specific school in Oklahoma to a statewide total of nearly two- 
hundred thousand students of programs in Florida including secondary, postsecondary, 
JTPA, corrections, and other programs. 

In most cases, the target group was a cohort of students, usually completers or 
graduates, from a particular academic year. Most frequently, the target group included 
postsecondary vocational education graduates wherein expectations for employment are 
strongest, that is, postsecondary programs are generally more occupation-specific where 
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secondary programs may tend to be both exploratory and occupation-specific. In New 
Hampshire, however, a five grade-level cohort (8th through 12th) was followed for four 
years. This study provided a longitudinal view of the interaction between work and 
educational experiences for high school youth. Some studies focused on all or a 
representative sample of students enrolled in vocational education during a specific time 
frame— not just program completers. In Texas and New Hampshire, the studies sought to 
analyze the labor market experiences for completers of different amounts of postsecondary 
vocational education or for those student enrolled in academic, general, or vocational tracks 
in high school. 

Since many of the studies were initial pilot efforts, extensive demographic data on 
students was not used In most instances, only the social security numbers were used in 
matching the UI wage records. However, several studies with more sophisticated analysis 
plans (Florida, Colorado, Virginia, Washington, and Wisconsin) were able to contrast their 
findings by gender, age, and ethnicity. Less frequently found were analyses based on 
proficiency in English, disabling condition, or economic status while enrolled. Analyses of 
employment experiences prior to or during enrollment in a community or technical college 
program were conducted only in Washington. When refined, such an analysis will provide 
a clear indication of the economic value of certain types of postsecondary vocational- 
technical education. 

The educational data and information used in these studies varied considerably, 
depending on the purpose and sophistication of the study. Most of the studies identified 
either the institution and/or the vocational-technical program in which students had been 
enrolled. In some of the studies, the Classification of Instruction Programs (CIP) code 
(used by the U.S. Department of Education) or a state or local program designation was 
used to indicate the specific type of vocational education the student had received. The CIP 
codes were used in Texas and Oklahoma in an attempt to examine the relatedness of 
training received and employment. In other states (Alaska, Indiana, and Missouri), the 
educational data was compiled to reflect the labor market experiences of students 
completing programs at secondary area vocational-technical centers (specialized schools) 
withoi^t regard to whether training was received in agriculture, business, marketing, or 
other fields. Similarly at the postsecondary level, aggregate numbers were compiled for 
associate degree or certificate completers. 
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Unfortunately, several of the reports which used aggregate data at some level above 
the local school cited major flaws and inconsistencies in the educational program data. 
Especially at the secondary level, local school officials are generally inconsistent in their 
designations of such basic elements as "vocational education student," "completer," or 
"leaver." In Arizona, which studied the earnings of postsecondary graduates across several 
institutions (both public and private), each institution was permitted to use its own 
designation of programs, which considerably limited the interpretation of the data. Without 
consistent and uniform definitions of vocational-technical education program completion, it 
is difficult to intelligently utilize labor market infokination in any analysis of program 
effects. If uniform and precise definitions are not used in these studies, analyses of 
transcripts should be considered to determine the nature and extent of coursetaking patterns 
and their relationships to labor market or further education experiences. 

The employment and earnings data found in the UI wage records were the 
centerpiece of the data collections in most of the studies. In Florida and Alaska, the UI 
wage records are supplemented with occupational title and the county of employment The 
UI wage record data of primary interest in the studies included work status (employed or 
unemployed), quarterly earnings (in some studies this data was aggregated over four or 
five quarters), the Standard Industrial Classification (SIC) code in which former students 
were working (a rough indicator of training r elatedness), mobility across industries over 
time, and the location of employment 

In addition to Ulwage record data, three states collect and compile information on 
the participation of former vocational education student/completers in higher or 
postsecondary education. Florida monitors enrollment of former secondary vocational 
education in community colleges and state universities. Colorado monitors whether or not 
postsecondary students have enrolled in other higher education programs and received 
degrees. In its pilot study, Indiana monitored the higher education enrollment of its 
secondary graduates. 

Analyses Conducted 

As noted in Table 16, most of the reports reflected analyses of how well program 
graduates performed in the labor market The common analyses reflected what percentages 
of the graduates were employed and their quarteriy or annual earnings. For the most part, 
these were descriptive reports compiled for statewide samples of graduates from selected 



71 



programs or, in some cases, all vocational-technical programs of a certain type (e.g., 
postsecondary) (Arizona, Colorado, Illinois, Washington, and Wisconsin), In three states 
(Alaska, Indiana, and Texas), the data was compiled and analyzed for selected schools or 
postsecondary institutions. 



Generally, the reports reflected the following analyses: 

Employment rates and quarterly earnings of associate degree graduates across 
various postsecondary vocational-technical programs (Arizona and Virginia) 

Earnings and employment patterns of vocational education students or graduates 
over time using longitudinal measures and comparing their performance to 
nonvocational students (Missouri, New Hampshire, and Wisconsin) 

Comparisons of completers and nonretuming postsecondary vocational-technical 
students in terms of percent employed, median or mean quarterly earnings, and 
inter-industry mobility of fomer students (Texas) 

Comparison of employment patterns and earnings of associate degree graduates 
with graduates of JTPA and other employment and training programs (Illinois) 

In Florida and Alaska, where occupational titles were also collected firom employers 
(in Horida, independent of the UI wage record), the training-related job placement 
rates for specific programs were calculated. In Washington and Oklahoma, 
analyses were conducted matching the industry of employment (SIC code) with the 
instructional program title (CIP) which the student had completed. Since 
vocational-technical education programs are occupation-specific rather than industry 
focused, the latter approach provides only an indirect meastire of training-related 
placement 

Analyses of employment pattems and income prior to, during, and following 
completion of a postsecondary vocational program provide an indicator of the extent 
to which such programs provide skill upgrading/retraining benefits for certain 
industries (Washington) 

Analyses reflecting the percentage of graduates or leavers who enrolled in 
postsc :ondary or higher education (Colorado and Florida) 




Z 

s 



a 

H 



6N 

OH 

25 ft. 
o 



o 

o 





g|l§| 






il ii 

2 9^ S 



Q 
H 

D 

2; 
O 

o 

< 



H 








0 § a 




P tS' « ^ - 
h g S g 

flu oD 



S S § 

g :r «3 a J5 

Iri III 



ge« « w C8 ^ Q 
4> O Q 

Sid? ss I 

5 a* i "f -S & 




00 



er|c 



75 



e 

o 



Xi 

d 

H 



O 

M 

H 

U 
M 



i 

H 
P 



OH 

o 



Q 



O 



Q 
O 

o 

u 

CO 

< 



CO 

H 

H 

CO 




15 Its 



00 
OQ 




! s S <« « I p 3 

) S B «> 2 § » i 




■S 
c 





1 

> 



IS 8 fell 
I a « .2 •§ •§ 

B.fi.S 



1 

00 



.S 




ERIC 



76 



This array of analyses illustrate the variety of comparisons of interest to researchers 
who, for the most part, were experimenting with the UI wage record database for the first 
time. The more sophisticated analyses were dependent upon crosswalks with other 
administrative databases such as the U.S. Department of Defense military personnel 
database or state higher education databases. The availability of the UI wage records 
enables researchers and evaluators to conduct a nimiber of highly cost-effective, reliable 
analyses which address important questions for program planning, development, and 
budgeting. 

The most useful analyses are dependent upon carefully constructed independent 
variables describing the type and amount of vocational education received For example, 
specific and uniform definitions of vocational education completers are necessary to 
accurately compare the employment rates of vocational and nonvocational education 
students statewide. Sorting out the effects and outcomes of vocational education programs 
by examining labor market information requires maintaining comprehensive student 
records, containing such information as current program and course titles, the amount of 
work experience credit completed, related academic courses completed, special services 
received (e.g., career planning and assessment) and special student status (e.g., disabled, 
disadvantaged, LEP), Labor market outcomes described in relation to these components 
provide precise and reliable direction for program improvement 



Conclusions 

In examining the findings, a number of general conclusions and policy lessons can 
be gleaned. The conclusions which follow offer insights regarding the general utility and 
coverage of the database, the future directions likely to be taken by states, local use of UI 
wage record information, technical concerns, uniform definitions, and training-related 
employment analyses. The final section of conclusions focuses on what these states have 
learned relative to general implementation (e.g., efficiency gains, regional [multistate] 
analyses, useful comparisons, and linking to other databases). 

Most of the studies from the thirteen states concluded that using UI wage records 
was a significant improvement over traditional follow-up studies in compiling information 
on the labor market performance of vocational-technical education graduates. Generally, 




mail follow-up surveys have low response rates and are plagued with potential response 
bias problems. While there are significant constraints in the UI wage data (most notably 
the lack of occupational titles of employment), the cost savings associated with the more 
comprehensive and longitudinal data set make the UI wage record data a valuable resource. 

In general, through the UI records, sixty-five to ninety percent of the graduates or 
former students were identified as in the labor market. As one might anticipate, the 
percentages offuimer students employed were higher for graduates of postsecondary 
programs than for secondary programs. In those states where higher education enrollment 
was matched, ^^^en to twenty percent of the former vocational education students were 
pursuing further education. Those studies examining multiple quarters found a higher 
percentage of graduates in the labor market than did studies which examined only one or 
two quarters of employment In Florida and Alaska where training-related placements were 
determined, approximately fifty percent of the graduates of postsecondary vocational- 
technical education programs were working in jobs for which they had trained. More 
specifically, in Florida, about eighty percent of postsecondary vocational completers are 
foimd. This data includes employment (about 70%) and continuing education (about 
25%). There is about a fifteen percent incidence of overlapping employment and 
continuing education. About seventy-five percent of the employment, or seventy-five of 
the seventy percent, is typically training-related. Training-related placement rates were 
lower for secondary level vocational education graduates. 

In the two states where pre- and postprogram earnings were compared, the 
graduates of postsecondary vocational-technical programs were earning more following 
their programs. In general, graduates of postsecondary vocational-technical programs were 
earning more than comparison groups and nonretuming students. 

Future Directions of States 

In general, the thirteen states reviewed in this study are each considering future uses 
for the UI wage records in evaluating vocational-technical programs. To date, four states 
(Alaska, Colorado, Florida, and Washington) have implemented provisions for ongoing 
use of the UI wage records. In two other states (Illinois and Indiana), the plans ai^ being 
finalized for implementing employment tracking of vocational education graduates using the 
UI wage records. While the remaining states report a generally favorable response to the 
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pilot testing or occasional reports for economic and/or political reasons, plans for the 
further development and use of the UI wage records remain stalled. 

The 1990 Amendments to the Perkins Act require state hoards to develop, in 
consultition with a Committee of Practitioners, a system of performance measures and 
standards for secondary and postsecondary vocational technical education programs 
receiving federal assistance. The final rules for the Perkins Act {Federal Register, "Rules 
and Regulations," 1992) allows recipients to evaluate annually the effectiveness of the 
entire vocational education program, regardless of which particular projects are assisted 
with federal funds. While this could be seen as an excellent opportunity for states to 
consider the use of the UI wage record data, it appears that only a handful of the thirteen 
states included in this study have moved to include the UI wage records in their 
performance assessment system. Only Illinois, Florida, and Washington definitely plan to 
use the UI wage record data. Washington will use the data for postsecondary measures 
and standards and possibly for secondary program measures if social security numbers can 
be obtained. Three states (Colorado, Missouri, and New Hampshire) have indicated that 
they will detmitely not use UI wage records, citing a preference for traditional follow-up 
studies and/or the lack of agency control of the UI data as major constraints. The 
remaining states are in various stages of considering the possibility of using the UI data. 

Beyond the new Perkins Act requirements for assessment in vocational education, 
several states were using or planning to use the Ui wage records as a broad indicator of 
education and training accountability. In Florida and Virgmia, the UI wage records have 
been used in analyzing education and training outcomes across several agencies, including 
JTPA programs, higher education, community college programs, corrections, and adult 
migrant programs. In Washington and Indiana, the recent creation of new state workforce 
preparation commissions/boards has spawned considerable interest in examining client 
outcomes across several education and training programs designed to improve the readiness 
of the state's workforce. In these states, the restructured policy and governance 
arrangements will supposedly facilitate greater coordination and place a premium on 
knowing more about client success in the labor market. State officials from each of the 
thirteen states in this study indicated there was a strong interest in using employment data 
as a general indicator of education and training effectiveness. 
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Local Use of UI Data 

In the states where institution-specific summaries of UI wage record data are 
available (Alaska, Colorado, Florida, Illinois, Indiana, and Washington), there appears to 
be considerable interest on the part of local administrators. However, in those states where 
it is possible to have the state UI wage record agency run special studies or specific 
analyses of the data for a given program or institution, only a handful of schools have 
requested such studies. In a few instances, state administrators interviewed for this study 
reported growing interest by local officials in using the matched employment and earnings 
data to evaluate programs. It appears that local administrators and policymakers are 
generally unfamiliar with the availability of this data and lack knowledge about the 
procedures for accessing and using the data in conducting local evaluations and decision 
making. States which move forward in expanding the use of UI wage records and other 
automated t^-itabases need to pay close attention to training administrators to ensure that the 
analyses and report formats used do indeed respond to the critical information needs of 
local policy boards and administrators. 

Technical Issues 

As with any other major database, a number of operational and technical issues are 
found in using and interpreting the infcnnation and data. Issues common to the UI wage 
records and their use in vocational education include the following: use of uniform 
definitions, confidentiality and corresponding safeguards, data reliability and accuracy, and 
data comprehensiveness. In the follow-up telephone interviews, these issues were 
explored with the state officials directly responsible for the UI wage record initiative 
pertaining to vocational-technical education. 

Confidentialiiy and Corresponding Safeguards 

In each of the states where work has been undertaken, the development of 
safeguards to protect the confidentiality of individuals and employers was a major concern. 
However, it is important to note that these concerns were successfully resolved and the 
studies were implemented. In no instance was the issue of confidentiality an 
insurmountable barrier. 

In some states, the state agency responsible for the UI wage record data has 
developed data sharing agreements which require the names of individuals and firms to be 
removed from all data, require cooperating agencies to produce a listing of social security 
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numbers for the population of interest, and provide cooperating agencies with requested 
data tables in which minimum cell size (e.g., 5 or 10 cases) are required. In some other 
states, the agency responsible for the UI data actually conducts the analyses and prepares 
the reports to assure that confidentiality standards are met 

In smaller states, it appears that the minimum cell size safeguards raise problems 
when attempting to examine the labor market participation and earnings of minority 
populations, which are originally small in number. 

Those states doing longitudinal analyses of the data while students were still in high 
school found it was necessary to obtain parental permission prior to accessing the 
iiiformation. Li both states, this requirement did not significandy diminish the sample size. 

While issues of confidentiality were successfuUy addressed in these state-level 
initiatives, they are likely to persist as states continue to look for ways to examine the 
effects of individual local programs. In small or rural counties, for example, if minimum 
ceU sizes are maintained for each of the industries reported in the state's data, it may be 
difficult to find a minimum of five or ten employers in manufacturing or simUar industries. 
As state and local officials move toward a more fine-grained analyses of the data, the issues 
of confidentiality will remain salient 

Data Reliability and Accuracy, and Data Comprehensiveness 

Several groups of workers are not included in the UI wage records in most states. 
Those excluded include workers who are self-employed, railroad employees, some 
agricultural workers, and employees of some nonprofit organizations. Among the thirteen 
states included in the study, some cite the lack of comprehensiveness as a problem. It 
appears that in larger states such as Texas or California which have economies that are 
predominantly rural and agricultural, or which have heavy concentrations of self-employed 
workers, these demographic patterns may present significant limitations to using the UI 
wage records. In other states, such as Rorida, where multiple databases are used, the 
prospects of matching nearly all (i.e., 95% or greater) of some program graduates is higher 
and thus the confidence in the data is higher in terms of predicting the postprogram labor 
market status of graduates. However, most states in the study did not indicate that the 
exclusion of some groups of workers from the UI wage records was a significant problem. 
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Some states have initiated traditional mail or telephone follow-up procedures to locate those 
former students not matched. 

State personnel who were interviewed reported a host of issues surrounding the 
validity and reliability of the UI wage record data in their state. The following list of issues 
arose in the discussions: 

• Some employers cite themselves as being in the SIC which has the lowest taxation 
rate. 

• One state estimates that perhaps ten percent of employers simply do not report UI 
wage record data and state-level monitoring efforts are not sufficient to reduce that 
number. 

• Another state reports that employers have as much as a ten percent error rate in 
entering social security numbers and other data. 

• In some states there appears to be as much as six months of lag time in receiving 
printouts from the agency managing the data system making it difficult to produce 
timely reports for local schools. 

States such as Florida that have worked for an extended period of time with UI 
wage record data have developed strategies for monitoring the quality of the data. These 
strategies include checking social security numbers against the Social Security 
Administration's list of unassigned numbers. In Florida, response analysis visits are made 
to a random sample of approximately twenty-five businesses annually to examine their 
payroll and reporting systems and to verify that the information gathered through matching 
and employee contact have been accurately reported and handled. 

Another salient issue is the availability and use of social security numbers (SSNs). 
Clearly, this is an issue focused largely on secondary level vocational education programs. 
In this set of thirteen states, most of the studies pertained to postsecondary vocational- 
technical education programs in which SSNs were readily available because these 
institutions used the SSN as a student identification number. However, not all 
postsecondary institutions require students to have SSNs. Obviously, to match UI wage 
records there must be a set of social security numbers provided by the educational 



ERIC 



82 



institutions. In states where the pilot or feasibility studies focused on high school 
vocational programs, it was reported that only one-third to one-half of the students had 
SSNs. In some states, state laws or state requirements for receiving vocational educativon 
funds require SSNs to be collected for high school or vocational high school students. 
When the latter strategy is used, making comparisons on the labor market involvement of 
vocational education students and nonvocational students is possible. 

Overall, most states in the study did not see this as a problem, hut argued instead 
that the quality and comprehensiveness of the UI wage record data war the strength of the 
data system. The UI wage records are not a panacea for evaluators and policymakers 
concerned with vocational-technical education. In each state, care must be taken in 
examining the quality and comprehensiveness of the UI wage record data. The problems 
of generating social security numbers for current and former students should also be 
considered by anyone planning to use the database. 

Uniform Definitions 

Perhaps the biggest challenge in using the UI wage record data lies in knowing 
specific information about the vocational-technical programs and/or the students who 
completed them The issues of how "vocational educatiorA programs" or "completers" or 
"vocational students" are defined is long-standing and complex. Some states have precise 
definitions that must be followed in local reporting in order for local schools or technical 
colleges to receive funds. In other states, schools can decide within general guidelines 
which courses and how many courses constitute the completion of a vocational program at 
the high school level. The problems of uniform definitions appear to be less complicated at 
the postsecondary level, but the issue still exists in several states. 

In the data reported in this study, it appears that the lack of uniform definitions was 
and is a significant issue in those states attempting to do statewide aggregation (i.e., to 
draw some conclusions about the effects of vocational education programs at the regional 
or state level). However, for states such as Alaska and Virginiii, where studies were run 
by the state agency upon the receipt of requests from specific schools or technical colleges, 
the assumption was made that these institutions can readily describe the specifics of their 
programs. In some states, the reports have cautioned readers against making comparisons 
across institutions because of uncertainties re^gaiding the content of the vocational-technical 
programs. 



In statewide analyses, separate governance of the vocational education programs 
(i.e., secondary versus communit>- colleges) creates lack of consistency in describing 
programs and students. As governance structures begin to change or realign themselves, 
one of the major concerns has focused on building common data systems, which would 
require adult education JTPA, and secondary and postsecondary vocational-technical 
programs to "work toward developing a common core of definitions and data reporting 
elements" (Paris, 1991, p. 24). 

In the states reported herein, the Committees of Practitioners and several inter- 
agency workgroups have been formed to study the issue, but it appears that only a few 
states (Colorado, Florida, and Washington) have made significant progress in building a 
set of uniform, state-level definitions. In general, the states who have worked most 
extensively on the matching of UI wage records understand more fully the problems of 
uniform definitions for vocational education. These states have developed various 
approaches for assuring quality of the information used to describe the vocational programs 
of the students and have taken precautions to make informed judgments about the labor 
market experience of students and their programs. 

Relatedness of Training to Employment 

Closely related to the issue of uniform data is the matter of linking the type of 
program completed to the use of acquired skills and knowledge in the labor market From 
the perspective of many labor economists, policymakers, and vocational educators, 
vocational-technical education programs only yield economic dividends for society if the 
trainees fully utilize the competencies gained in related jobs. Historically, employment in a 
training-related job has been the dominant criterion for judging the effectiveness of 
vocational-technical education at both the secondary and postsecondary levels. However, 
this issue has been confounded by the recent Perkins legislation which charges vocational- 
technical education programs with a broadened emphasis on orcupa ^nal and academic 
skill development. Others within the field have contended for nearly a century that the 
purpose of vocational education is broader than preparing people for specific occupations 
and encompasses developing general employability; consumer and homemaking skills; 
career awareness and exploration; broad understanding about business, technology, and the 
economy; and avocational interests. 
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Since individual vocational programs are structured around occupations (e.g., 
nursing) rather than industries (e.g., health services), it is technically impossible to 
determine the precise training-related match unless occupational titles are collected from 
employers. In Rorida and Indiana^ occupational titles are collected from employers on a 
regular basis through supplemental or follow-up surveys. In Alaska, the collection of 
occupational tide was mandated by state legislation and has been part of the UI wage record 
system for several years. 

Six states have attempted to analyze and report employment patterns by the SIC 
codes and relate them to the CIP codes used by the U.S. Department of Education. With 
the exception of Washington, the respondents indicated dissatisfaction with this analysis. 
Agency personnel in Washington have developed a protocol which interfaces eighteen CIP 
clusters with various SIC codes. The reports provided to community and technical colleges 
provide an indication of the percentage of former students working in training-related jobs. 
In one state, the reporting of high percentages of graduates as employed in "eating and 
drinking establishments" caused considerable controversy. Another state has suggested 
that additional crosswalks be developed through which industry codes and occupational 
codes could be linked and the resulting occupational codes aligned with CIP codes for 
vocational-technical programs. 

In general, the vocational education researchers and state officials who have been 
involved in these analyses cite the inability to make an adequate link to the training- 
relatedness issue as the major limitation of the UI wage record data. Without state- 
mandated collection of occupational title data or valid interpretive crosswalks, most 
arguments do not support widespread use of the UI wage record data to answer the key 
questions about the contribution of vocational programs to economic productivity. 
However, the limitations must be considered u\ light of the broadening mission of 
vocational education in many states and communities. 

General Implementation Consicerations 
Efficiency Gains 

One of the major benefits of using the UI wage records is the anticipated savings 
over the high costs of collecting follow-up information through surveys. The Office of 
Technology Assessment (1989) reports tiiat twenty-seven states use follow-up surveys to 
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collect data on job tasks, employers' assessment of job performance, as well as information 
on wages. In four of the states reviewed in this study, requirements to conduct follow-up 
surveys were eliminated or suspended. In one state it was noted that nearly twenty percent 
of the commxmity and technical colleges continue to conduct follow-up information via 
surveys which ths^y find particularl} useful. Two additional states indicated they planned to 
eliminate the follow-up requirement for local schools or postsecondary institutions. The 
remaining states indicated that state-level follow-up requirements were either non-existent 
or implemented in a highly permissive maimer by the state board for vocational education. 

Clearly, there has not been a rapid, national shift to replace follow-up surveys with 
automated database matching in most states, in contrast to what some projections in the 
mid-1980s suggested 

Reaching Adjacent States 

Since mobility factors are prominent in the workforce, several states have examined 
the feasibility of sharing UI wage records. The ability to locate graduates and former 
students in adjacent states is paiticularly crucial in small states (e.g.. New England) and in 
large metropolitan areas that span state borders (e.g., St Louis-Kansas City, New York- 
New Jersey-Philadelphia). 

Washington has developed data-sharing agreements with Alaska, California, Idaho, 
and Oregon and monitors regional mobility of graduates on a regular basis. Other states in 
the sample (Colorado, Florida, and New Hampshire) have pursued discussions with 
surrounding states, but report limited success in reaching the point of developing data- 
sharing agreements. The remaining states had only begun to consider this possibility. The 
problem is complicated by incompatible UI wage data and states having different views on 
the utility of shared UI wage record data. 

Comparisons and Analyses 

As noted earlier, the emplojmient and earnings data available in the UI wage records 
only have utility when they are interpreted in relation to specific types of students or 
educational programs. Using the UI wage record data as outcome measures for vocational- 
technic^ education programs, the states in this study made a variety of comparisons and 
analyses as follows: 
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• Missouri and New Hampshire compared secondary vocational education 
students/graduates with other groups of high school students/graduates over a five- 
year longitudinal period to determine the extent to which vocational education 
courses/programs accounted for differences in employment rates, industry of 
employment, and earnings. Other states in the study indicated the longitudinal 
tracking capabilities of the Ulwage records are valuable for understanding the long- 
term effects of programs upon labor market performance. 

• Several states (Colorado, Florida, and Washington) have compared the employment 
patterns to standards that were legislatively mandated for vocational education 
programs or training-relaxed job placement rates that emerged from traditional 
follow-up studies. The expectation here is that programs with low training-related 
placement rates should be phased out and considerable savings in public costs are 
realized. 

• States are also interested in drawing comparisons for different types of students. 
Understanding the extent to which labor market outcomes are different for 
male/female, low-income, disabled, incarcerated, and unemployed/employed adults 
can assist policymakers and educators in developing appropriate support services 
for these groups while they ate enrolled in vocational-techri ical programs. Further, 
such analyses are essential in informing policies and priorities for other human 
service agencies such as the Job Service, Welfare, and Vocational Rehabilitation. 

• In Washington, the labor market performance of adults was compared prior to, 
during, and following their participation in various community and technical college 
programs. Such analyses provide rough proxies for determining the economic 
productivity effects of job-specific training programs in particular industries. 

Linking to Other Databases 

Beyond the UI wage records, several states have identified other automated 
databases which can be accessed using social security numbers. Florida is clearly the 
pioneer in this arena, having developed a system for linking with data files in higher 
education, military services, higher and adult education, corrections, and federal 
employment. Other states have been able to link social security numbers of students and 
graduates with databases m higher education, military personnel, and state income tax 
records. 
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States involved in multi-agency matching processes note that the negotiations are 
generally long and difficult, but that significant cost savings can be realized along with 
locating much higher percentages of former students. Such efforts also provide at least 
tacit endorsement for expanding interagency collaboration to improve public policy in 
education, training, and human resource development 



Recommendations 

The findi.igs and conclusions from this analysis of thirteen states and their 
experiences with the UI wage record database suggest a number of recommendations. 
These recommendations can be grouped and disctissed as actions needing to be taken by 
policy analysts and leaders, and potential initiatives for researchers. 

Policy Development Concerns 

At the federal and state level, a number of actions should be considered for 
expanding and improving the use of UI wage record data in vocational-technical education. 

First, all State Directors of Vocational Edu :ation, in collaboration with their 
Committees of Practitioners, need to seriously consider using the UI vage record database 
as they formulate state systems of performance measures and standards. The evidence 
collected in this study suggests that the UI wage records contain useful information for 
judging whether or not graduates and former students are in the labor market and what their 
earnings look like. While the issue of whether or not their employment is related to the 
type of vocational education they rexieived cannot be answered directly, it is possible to 
supplement the UI wage record data, as Horida has done, through sample and population 
surveys of employers to coUect information describing job tides, location of emploj-ment, 
and perhaps employers' satisfaction witii tiie training received. Beyond tiie UI wage record 
database, state directors and the agency staff should carefuUy examine possible data linking 
witii systems of postsecondary and higher education institutions, as well as otiier federal 
databases (e.g., tiie Department of Defense, Postal Service, federal personnel). States tiiat 
have linked witii other database;s have been able to successfully locate higher percentages of 
former students and acquire useful information describing tiieir postprogram status (i.e., 
working, school, or botii), geographic mobility, and income. 
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Second, state officials and local administrators need to develop high quality student 
record data. In order to access the UI wage record data and obtain useful analyses, social 
security numbers must be collected and recorded before students leave the program. 
Equally important, uniform definitions of vocational education programs, students 
(including age, gender, special student status), completers, leavers, and support services 
provided must be utilized Without consistent and uniform data on program demographics, 
there are few possibilities for meaningful analysis. If comparisons are to be made on a 
statewide basis to identify programs or conununities needing assistance, it is impossible to 
make that judgment imless the characteristics of the program or students are known. For 
example, one may find that urban vocational programs have lower employment rates than 
suburban vocational programs. To adequately interpret the meaning of such a finding, one 
must consider the ability levels of the students, the amount and type of vocational 
instruction and support services they received, the economic conditions of the community, 
and a host of other factors. Hence, it is important to recognize that acc>:;ss to UI wage 
records is only one aspect of using and interpreting the data appropriately. 

Third, to reduce some of the administrative costs and to enhance the quality and 
comprehensiveness of the data set, states should consider forming regional consortia. 
These regional consortia, which should be modeled around natural labor markets, could 
develop interstate data-sharing agreements that would facilitate the building of 
comprehensive folio w-up systems using UI wage records as well as other federal and state 
data systems. 

Fourth, efforts are needed to improve the interpretative power of the UI wage 
record data. More specifically, states should develop special protocols for the SIC codes 
which allow the industry of employment data to match closely with specific vocational 
education programs completed by fomier students. The state of Washington has developed 
such a protocol. At the federal level, crosswalks need to be developed or refined which 
permit the SIC codes to be linked to appropriate codings for instructional programs such as 
the U.S. Department of Education's Classification of Instructional Programs (CIP). 

Fifth, the U.S. Department of Education, in cooperation with other federal 
agencies, must provide greater leadership aimed at building greater awareness and 
utilization of the UI wage records and other automated databases toward effective 
evaluation of vocational-technical education programs. In June 1992, the National 
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Commission for Employment Policy recommended to the U.S. Department of Labor that 
the national methodology for performance standards in job training programs funded under 
JTPA be modified to incorporate the extensive use of UI wage data. In follow-up to this 
recommendation and other reporrs, the newly enacted Job Training Reform Amendments of 
1992 will require the Conmiissioner of Labor Statistics to 

determine appropriate procedures for establishing a nationwide database 
containing information on quarterly earnings, establishment and industry 
affiliation, and geographic location of employment for all individuals for 
whom such information is collected by the States. (Section 405) 

With leadership provided by the Department of Labor and other federal departments, it is 
imperative the Office of Vocational and Adult Education pursue similar efforts on an 
interdepartmental basis. More specifically, the pilot demonstration projects in several states 
or regional consortia of states should be considered and supported. Tli^se pilot 
demonstrations should incorporate many of the policy recommendations cited above and be 
rigorously evaluated in terms of their data quality, effectiveness in providing useful 
information for performance measures and standards, and cost effectiveness. 

In addition, the Office of Vocational and Adult Education should be providing 
active support for pilot demonstrations through the sponsorship of conferences and 
workshops for appropriate state officials and local administrators, as well as the 
development of technical assistance guides and procedural manuals. In these efforts, states 
such as Florida and Washington who have developed sophisticated labor market 
monitoring systems should be highlighted. The various agencies involved in managing 
personnel data systems at the federal level should provide both encouragement and 
assistance to states in developing appropriate uses of this data if they expect to see 
definitive returns on the investment of federal funds for education, training, and economic 
development 

Research, Development, and Demonstration: Some Possibilities and 
Prototypes 

The richness of the UI wage record data and its potential for use in assessing the 
quality and effects of vocational-technical education is, at best, only minimally understood 
by professionals, administrators, and policymakers. Responses to a wide variety of 
questions can be developed beyond those pertaining directly to the labor market 
performance of former students. Appendix C provides two tables illustrating studies or UI 
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wage record reports which would address state-level policy issues (Chart A) and local 
program improvement issues (Chart B). In each of these charts, the specific UI and 
educational data that should be linked via social security number are suggested. The 
comparisons or analyses to be drawn from the resulting data are indicated, along with 
potential uses of this information in policy development and program improvement 

State-level policymakers and legislators would likely find longitudinal and 
comprehensive analyses of the earnings and employment of vocational education students 
helpful in informing several public policy concerns. These include justifying funding 
increases and program approvals for high wage, high demand technical programs; 
balancing supply and demand for skills in specific industries; siting new programs for 
regional labor markets that are experiencing growth; pkxming new initiatives to reduce 
income differentials by gender .Jid race; and informing employers, the general public, 
parents, and prospective students of the economic value of vocational-technical education 
programs. Other uses for these analyses could be improving financial aid policies for skills 
training, creating mcentive programs for business and industry to invest in skills upgrading 
programs, and determining the appropriate level of public investment in vocational 
preparation at the secondary and postsecondary levels. Making these analyses will be 
dependent, of course, upon uniform statewide educational data which describe reliably the 
program of enrollment and other key information that enables informed recommendations 
to be drawn for policies regarding vocational education and economic development 
programs. 

Demonstration projects should be undertaken by state boards of vocational 
education in collaboration with other state agencies (especially those agencies administering 
JTPA programs) and with support from the U.S. Department of Education, to develop and 
test the feasibility of linking statewide educational data to the UI wage records. As 
suggested by Chart A, this might include studies which examine a number of critical 
questions cited above. Further, demonstrations of the use of this data to address outcome 
questions will produce us-^ful mechanisms for the state's system of performance measures 
and standards. At least one or two state demonstration initiatives should involve regional 
consortia. Regional demonstrations are of critical importance where major urban 
communities and regional labor markets span state boundaries (e.g., St. Louis, Kansas 
City, New York/New Jersey). 
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LocaMevel demonstrations of the UI wage record data systems are equally 
important for addressing the issues of continuous quality improvement As school systems 
and postsecondary institutions move toward total quality management, the importance of 
informed data on the performance of students, graduates, and programs is crucial to 
effective decisions. Further, a broad array of local innovation and experimentation is the 
key to developing appropriate and optimally usefiil state-level data and information systems 
employing the UI wage records. State systems should, in large part, emerge from the most 
successful applications and demonstrations at the local level. 

Chart B in Appendix C illustrates several management questions that should be 
addressed in local demonstration projects. To ensure that program and institutional 
accountability remains high, administrators need to know the extent to which their 
graduates are in the labor market, in training-related industries and occupations, as well as 
their earnings over an extended period of time. With uniform and comprehensive 
educational daia, a number of other key questions can also be examined regarding specific 
innovative programs such as Tech Prep or Youth Apprenticeship. The important questions 
pertaining to earnings and employment and their relationships to overall academic 
achievement, financial aid, full- or part-time student status, and mobility of graduates can 
also be addressed. When compiled locally on the performance of former students, these 
reports provide invaluable insights for prospective students, parents, and high school 
guidance counselors who are advismg students on the enrollment choices for secondary or 
postsecondary programs. 

States considering the expanded use of UI wage records should support local 
demonstration projects. Among other funding sources, such projects could be supported 
from the federal Perkins allocation that each state receives for statewide leadership projects. 
State general purpose revenues and JTPA funds might also be aJIocated to demonstrate the 
impact of interagency cooperation. 

Finally, it is imperative that the Department of Education provide mechanisms for 
monitoring and networking the various state and local UI wage record demonstrations. 
Such efforts coordinated through NCRVE, the National Occupational Information 
Coordinating Committee, or other appropriate organizations will enhance the development 
and exchange of effective practices, computer-based programs, and reporting formats. 
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APPENDIX A 

FEDERAL GUffiELINES FOR INCOME AND ELIGIBILITY 

VERIFICATION 



States have a degree of latitude regarding what is included in its UI wage record 
file. Although minimum data requirements are established by the federal government, 
states are free to collect it in such a manner that meets their individual needs. In addition, 
some states collect more information than is required at the federal level. The following is 
excerpted from the U.S. Department of Labor's UIPL 47-86, describing the procedures to 
be used in collecting and processing the minimum income and eligibility verification under 
the Deficit Reduction Act of 1984 (P.L 98-369). The following paragraphs and tables 
describe what might be available in individual states. Persons wanting greater detail of 
their state's system should contact their state UI agency. 



Attachment to UIPL No. 47-86 

I. Purpose 

This document outlines the requirements and procedures for use of standardized formats as 
required by the income and eUgibiUty verification procedures of the Deficit Reduction Act 
of 1984 (P.L. 98-369). The requirements, procedures, and formats are designed to 
provide uniform definitions, consistent data elements, and uniform standards to faciUtate 
the exchange of information wiUiin and between states. 



n. Authority 

The Deficit Reduction Act of 1984 requires state agencies tiiat administer the Aid to 
Families witii Dependent Children (AFDC) program, tiie Adult Assistance programs (in the 
Territories), the Medicaid program, the Food Stamp program (FSP), and the 
Unemployment Compensation (UC) program to develop an Income and Eligibility 
Verification System (lEVS) meeting certain statutory requirements. Under tiie statiite, state 
agencies "must adhere to standardized formats and procedures established by tiie Secretary 
of Healtii and Human Services (in consultation witii tiie Secretary of Agriculture)" to 
exchange information to establish or verify eligibiUty. The fmal regulations implementing 
Uiex requirements were issued on February 28, 1986 (51 FR 7178). 
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in. Requirements 

A. The requbements of this section ^ply to state agencies administering the following 
programs: 

1 . The FSP under the Food Stamp Act of 1977, as amended; 

2. The AFDC program under title IV- A of the Social Security Act; the Adult 
Assistance programs under titles I, X, XIV, and XVI (AABD) of the Social 
Security Act; 

3 . The Medicaid program under title XIX of the Social Security Act; 

4. The UC program under title HI of the Social Security Act; and 

5. The State Wage Information Collection Agency receiving quarterly wage 
reports from employers as required by section 1137(a)(3) of the Social 
Security Act 

B . State agencies shall adhere to the following requirements when they exchange EVS 
information for purposes of conducting computer screens, computer matches, or 
other types of automated analyses to verify eligibility and benefit payments. These 
requirements must be reflected in any related agreements executed by the involved 
State agencies: 

1 . State agencies shall use the data element definitions included in the Duta 
Element Dictionary for all ex'^hanges and shall modify existing definitions to 
meet this requirement 

2. In responding to requests for data. State agencies shall provide an output 
record in the appropriate prescribed 480 character record, and shall include, 
at a minimum, all tiie critical data elements (those elements labeled "MUST 
NOT BE OMITTED" in die Data Element Dictionary); except tiiat (1) 
agencies having existing exchange systems in operation may continue to use 
such systems so long as such exchanges are agreeable to both parties and 
the critical data elements are included, and (2) agencies establishing direct 
on-line access between requesting and responding agencies may develop 
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screens and output reports consistent with the capabilities of the medium so 
long as the critical data elements are included. 



3 . State agencies requesting information shall include sufficient data to identify 
individuals in the responding agency's files. At a minimum, this shall 
include the name and Social Security number of the individual for whom the 
exchange is being conducted; except that agencies shall accept information 
in the appropriate 480 character standardized format if requested to do so. 

IV. Technical Documentation 

The following tabs set forth the technical requirements established for state use in 
exchanging information for income and eligibility verification. Tab A sets forth the 
systems pn>cessing and exchange requirements the states must follow. Tabs B, C, and D 
present the standardized formats for data exchange among state public assistance agencies 
(Tab B), wage information collection agencies (Tab C), and imemployment compensation 
agencies (Tab D). Tab E contains the Data Element Dictionary and requirements. It 
includes a table of critical data elements, by the match record in which they appear. The 
data element nimiber is included to provide a convenient cross-reference to the Data 
Element Dictionary. This table should help states identify the critical data elements and 
incorporate them into the state data exchange systems. 

Standardized Formats 

To facilitate the exchange of information between programs, standardized formats 
and data elements have been developed in the following three areas: 

• Assistance Programs — Match Record Type 1 

• Wage/Earnings — ^Match Record Type 2 

• Compensation/Benefits — ^Match Record Type 3 

The formats represent the complete file layouts and data elements needed to 
exchange information on clients from public assistance agencies, wage information 
collection agencies, and unemployment compensation agencies. 

Each format contains the following four types of data elements: 
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Identifying information assures the unique identity of the individual in the record. 
Basically, this information consists of name, Social Security number, address, 
agency case number, date of birth, and sex. 

Case/Eligibility/Client data includes information from the case record which is used 
to determine eligibility for and the amount of benefits. 

Wage/Benefit/Resource information includes the data from employers and programs 
needed to compare client reported information such as quarterly wages, employers, 
and unemployment insurance benefits. 

Reports/Control information to establish records controls or report conditions such 
as office numbers, codes, and match conditions. 



TAB C: Wage/Earnings— Match Record Type 2 (MR-2) 

The Wage/Earnings standardized format is prepared by the State Wage Information 
Collection Agency (SWICA) in response to a request from either a public assistance agency 
or an unemployment compensation agency. The Wage/Earnings format, file layout, and 
technical description are described in the pages following. The fonnat con ams information 
such as employer name, employer address, employee work location, and reported wages 
for the most recent available quarter and an additional four calendar quarters. For match 
hits that identify multiple employers, one full 480 character output record must be used for 
each employer. Some of the data elements included in the format are critical to properly 
develop the match. Otiier data, altiiough not critical would enhance the ability of tiie 
requesting agency to process the matched data and conduct necessary verification activities 
and case actions. 
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APPENDIX B 
TELEPHONE INTERVIEW PROTOCOL 



Review of State Summary 
This draft summary will be included in the Technical Report as presented herein xmless 
modified. 

1 . Is the enclosed statement an accurate and comprehensive overview of the various 
activities associated with UI wage record study or initiative in your state? 

2. Do you have any additions, corrections, revisions, or refinements to offer or 
suggest? 



State-Specific Questions 

1 . The summary of your state is based upon information contained on an 
attached document. Have there been any new documents issued by your or 
any other state agency on the use of UI wage records as a tool for vocational 
education? Have any of the following documents been updated? 

Comments: 



2, Do you anticipate any future activities in your state using the UI wage 
record data to follow-up clients of vocational education? 

A. To what extent (or how) will the UI wage record data be used in the 
System of Performance Standards being developed and required 
under the Perkins Act? 

B. Has the UI wage record data been discussed as a data source for 
new or expanded state educational accountability measures (e.g., 
school report cards)? 

C. Have any eligible recipients (LEAs or Postsecondary Institutions) 
developed strategies for using the UI wage record data for local 
evaluation or assessment? 
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D. 



To what extent, if any, do you envision utilizing UI wage record 
data in consort with other training institutions/programs (JTPA, 
welfare, proprietary schools, corrections. . •)? 



3. As you look into the future over the next five years, what reasonable goals 

could your state agency adopt, if any, for the use of the UI wage record data 
on vocational-technical education? 

Other States 

Attached is the listing of the thirteen states that are included in our study. Are you aware or 
familiar with other significant activities in states that are not included on this list? 

General Issues 

Through our review of the literature and in conversations with others in the field, the 
following list of issues may or may not be of concern as vocational educators utilize the UI 
wage record system. Has your state addressed or considered these issues and, if so, how? 

ISSUE YES NO 

1 . Personal assurances of confidentiality 
If yes, how? 

2 . Reporting safeguards for confidentiality 
(e.g., minimum cell sizes) 

If yes, how? 

3. Employees not included in the UI Wage Record Data Base 
(e.g., self-employed workers) 

If yes, how? 
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ISSUE 



YES NO 



4. Reliability and accuracy of the UI Wage Record Data Base 
If yes, how? 

5. The relationship of Education/Traimng to Job Placement 
(e.g., CIP/SIC match) 

If yes, how? 



6 . Availability of student social security numbers 
If yes, how? 



7 . Uniform definitions of vocational education 
(e.g., programs, courses, and completers) 
If yes, how? 



8. Adding to the database (e.g., asking employers for 
additional information) 

If yes, how? 

9. Appropriate and useful comparisons (e.g., comparing 

data to other groups or to previous year's follow-up studies) 
If yes, how? 



10. Elimination or reduction of the existing follow-up studies 
or reporting systems 
If yes, how? 



1 1 . Follow-up on nonmatched students in the UI Wage Record File 
If yes, how? 
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12. Discussion with surrounding states to identify their residents 
working in your state 
If yes, how? 



13. Sub-state geographic analyses of the data 
If yes, how? 



14. Matching student record data with other databases 

(e.g., military, higher education, and federal personnel files) 
If yes, how? 



15. Comprehensiveness and quality of student records files 
(e.g., courses, test scores, and demographic information) 
If yes, how? 



1 6. Hours worked in the quarter — part-time versus full-time 
employment 
If yes, how? 



1 7 . Inteq)reting data appropriately in light of economic and 
demographic conditions. 
If yes, how? 



18. Other issues: 
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APPENDIX D 
STATES, DOCUMENTS AND CONTACT PERSONS 



Alaska 



Documents Reviewed: 

"Employment Status and Earnings of Alaska Vocational Technical Center Training Program 
Graduates Completing Training in Fiscal Year 1988," Jeff Hadland, Economist, Alaska 
Department of Labor, June 30, 1990. 

"A Comparison of Alaskan Female and Male Wage Income and Employment Patterns in 
1988: The Gender G^," Alaska Department of Labor, August 1990. 

"Employment Status and Earnings of 1987 Vocational Program Completers, Martin Luther 
King Career Center, Anchorage, Alaska," JoAnn Wilson and Jeff Hadland, Alaska 
Department of Labor, and Flory Vinson, University of Alaska at Anchorage, September 6, 



"Employment Status and Earnings of Alaska Vocational Technical Center Training Program 
Graduates Completing Traming in Fiscal Year 1989," JoAnn Wilson and Jeff Hadland, 
Alaska Departmf nt of Labor, September 1 8, 199 1 . 



Contact Person: 

Ms. Brynn Keith, Executive Director 

Alaska Occupational Information Coordiaating Committee 

P.O. Box 25501 

Juneau, AK 99802-5501 

(907) 465-4500 

FAX: (907)465-2101 



1991. 




Arizona 



Document Reviewed: 

•"A Study of Public and Private Post Secondary Institution Vocational Education Program 
Completers," Robert V. Kerwood and Sid Borcher, Arizona Center for Vocational 
Education, Center for Excellence in Education, Northem Arizona University, Flagstaff, 
Arizona, June 1986. 

Contact Person: 
Mr. Ted Davis 

Arizona Department of Education 
1535 W. Jefferson 
Phoenix, AZ 85007 
(602) 542-5566 

Colorado 

Documents Reviewed: 

"A Longitudinal Tracking Study of Short-Term Education and Employment Outcomes of 
Colorado Community College Graduates," Gregory P. Smith, Colorado Community 
College and Occupational Education System, December 6, 1989. 

"Colorado Commission on Higher Education Accountability Newsletter," May 1991. 

Colorado House Bill No. 1 187. 

Contact Person: 

Gregory P. Smith, Coordinator of Institutional Research and Evaluation 
Colorado Community College and Occupational Education System 
1391 N. Speer Boulevard, Suite 600 
Denver, CO 80204-2554 
(303) 620-4034 
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Florida 

Documents Reviewed: 

"Florida Education and Training Placement Information Program, 1988-1989 Summary of 
Findings,** Annual Report, Florida Department of Education, May 1989. 



"Rorida Education and Training Placement Information Program," Annual Report, Florida 
Department of Education, August 1990. 



Contact Person: 

Jay J. Pfeiffer, Systems Project Director 

Florida Educational and Training Placement Information Program 

137 W. Gaines St 

Tallahassee, FL 32399 

(904) 487-0900 



Illinois 



Documents Reviewed: 

"A Primer on the Employment Tracking System," Illinois Department of Employment 
Security, 

"Employment Tracking System: User Guide," Illinois Department of Employment 
Security. 

"On-The Job Training Programs in JTPA: A QuaUty of Service Report to tiie nUnois Job 
Training Coordinating Council," Illinois Department of Commerce and Community 
Affairs, October 24, 1990. 



Contact Person: 
Mary Ann Merano 

Department of Adult, Vocational, and Technical Education 
Illinois State Board of E<Iucation 
100 N. First Street 
Springfield, IL 62777 
(217) 782-4870 




Indiana 



Documents Reviewed: 

Agenda Items from May 1 1, 1989, and June 7, 1990, Meetings of the Indiana Commission 
on Vocational and Technical Education. 



Contact Person: 

Linda S. Piper 

Indiana Commission on Vocational and Technical Education 
309 W. Washington Street 
Indianapolis, IN 46204 
(317) 232-8528 
FAX: (317)232-1815 



Missouri 

Documents Reviewed: 

"The Longitudinal Follow-up of 1981 Missouri High School Graduate," Final Report, 
Missouri Testing and Evaluation Sendee, College of Education, University of Missouri at 
Columbia, October 1987. 



"The Effectiveness Index Formula For Incentive Funding," Missouri Department of 
Elementary and Secondary Education. 



"Employment Security Report 1981-1986," Missouri Division of Employment Security. 



Contact Person: 
Bill Brandt 

Missouri Department of Elementary and Secondary Education 
Box 480 

Jefferson City, MO 65102 
(314) 751-6875 
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New Hampshire 
Document Reviewed: 

"Working and Schooling Decision: A Study of New Hampshire Teenage Labor Market 
Behavior and the Level of Educational Attainment," Third Report of "The School-to-Work 
Transition: The Role of Unemployment Insurance Wage Credit Records in Longitudinally 
Tracking Youth from School to Work," New Hampshire Employment Security, June 
1991. 

Contact Person: 

Kenneth E. Yasuda, Labor Market Analyst 
Economic and Labor Market Information Bureau 
New Hampshire Department of Employment Security 
32 S. Main Street 
Concord, NH 033012 
(603) 228-4174 

Oklahoma 

Document Reviewed: 

"Using Employer Wage Records To Determine Vocational Completer Labor Market 
Outcomes," Richard C. Makin and Curtis G. Shumaker, Oklahoma Department of 
Vocational and Technical Education and the Oklahoma Occupational Information 
Coordinating Conmiittee, August 22, 1990. 

Contact Person: 

Curtis G. Shumaker 
Oklahoma SOICC 
1500 W. Seventh Avenue 
StiUwater, OK 74074-4364 
(405) 743-5198 

Texas 

Documents Reviewed: 

"Quarterly Employment Tracking System: Unemployment Insurance Wage Record 
Feasibility Study and Pilot Test Project," Final Report, Texas Occupational Information 
Coordinating Committee and Andersen Consulting, Austin, Texas, July 1990. 
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"Pilot Study Findings of the Use of the Texas Employment Commission Unemployment 
Insurance Wage Records to Document Outcomes of Technical- Vocational Education and 
Traimiig," Texas Higher Education Coordinating Board, Austin, Texas, May 1991. 



Contact Person: 

Richard Froeschle, Executive Director 

Texas Occupational Information Coordinating Committee 

15th and Congress 

TEC Building, Room 526T 

Austin, TX 78701 

(512) 463-2399 

FAX: (512)475-1260 



Virginia 



Document Reviewed: 

"Tracking Education Completers," Ann D. Lang, Planning Research Economist, Virginia 
Employment Commission, August 1988. 



Contact Person: 

Delores Esser, Director 
Economic Information Services 
Virginia Employment Commission 
703 E. Main Street 
P.O. Box 1358 
Richmond, VA 23211 
(804)786-7496 



Washington 



Document Reviewed: 

"Vocational Outcomes in Washington Community Colleges," Baseline Report, A Report 
Prepared by the State Boaxd for Community College Education, October 1990. 



Contact Person: 
Loretta Seppanen 

Washington State Board for Community College Education 
319 7th Avenue, MS: FF-11 
01yrapia,WA 98504 
(206) 753-3685 
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Wisconsin 



Document Reviewed: 

"A Longitudinal Study of Earnings of VTAE Graduates," An Analysis of Earnings of 
1982-83 Wisconsin PostSecondary Vocational/Technical Education Program Completers 
Utilizing 1985 State Income Tax and 1988 Wage Reporting Data Compared to Adjusted 
National Census Data, Merle E. Strong and Daniel Jarosik, Vocational Studies Center, 
School of Education, University of Wisconsin at Madison, 1989. 



Contact Person: 

Daniel Jarosik 
Vocational Studies Center 
University of Wisconsin at Madison 
964 Educational Sciences Building 
1025 W. Johnson Street 
Madison, WI 53706 
(608) 263-3696 
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